Entropy

The average surprise in a distribution — how many bits it takes to encode an outcome drawn from it. Zero when one outcome is certain, highest when everything is equally likely.

A definition is not an understanding. This is taught properly in Entropy, cross-entropy, KL and perplexity, part of Probability & statistics.

Learn this properly