Why Linear Algebra Critical to Understand Deep Learning..?
nformation theory is a measurement of the quantity of information. It consolidates
calculus, probability, and statistics which are mathematical roots of deep learning.
It is all about how we represent information in the form of:
● Few bunch of letters on a paper
● Visual signals
Throughout history, humans not only evolved but also our way to represent information. The way we communicate today is changed significantly. In order to make a machine
understand the information, we must know how to describe information with a
The significance of Information theory:
In a language translator, there are 2 major components:
1. Encoder: Takes input in a language and converts it into vector.
2. Decoder: Take vector as input and translates it into a target language.
In order to follow semantic and syntactic rules while translation it must keep track of
objects and their relationship in the input signal.
This is the part where information theory plays important role. Here Information measures decides our training objectives. If we have less information then the next question arises how much information is required?
Few widely used and advanced applications of deep learning and information
1. Noise removal in communication signals
2. Theory of GAN
3. Natural Language Processing
4. Image Captioning