This is the part where information theory plays important role. Here Information measures decides our training objectives. If we have less information then the next question arises how much information is required?
Few widely used and advanced applications of deep learning and information theory:
1. Noise removal in communication signals 2. Theory of GAN 3. Natural Language Processing 4. Image Captioning