Entropy and Uncertainty

Abstract

We give a survey of the basic statistical ideas underlying the definition of entropy in information theory and their connections with the entropy in the theory of dynamical systems and in statistical mechanics.

Keywords

• Entropy
• relative entropy
• uncertainty
• information theory