Information theory MOC

Shannon information

The Shannon information or surprisal of a discrete random variable is the random function info

where corresponds to the unit , corresponds to the unit , and corresponds to the unit . Up to choice of , the Shannon information function is the unique function satisfying the following properties:

1. If then , i.e. a certain event is perfectly unsurprising.
2. If then , i.e. the more unlikely an event the more surprising.
3. If two independent events are measured, the total Shannon information gained is the sum of the Shannon information of the individual events.

Proof

proof

---

develop | en | sembr