Atlas record: https://www.aitimeline.dev/en/events/evt-0064/
Research · July – October 1948
A Mathematical Theory of Communication
Shannon defined information quantitatively and introduced entropy as a measure of a source's uncertainty.
Why it matters
Prediction acquired a unit it could be measured in, the basis of all later language modelling.
The paper solves an engineering problem, sending messages through a noisy channel. What matters here is a side result: modelling a source as a sequence of symbols with probabilities, and approximating text with n-grams. It is the first quantitative account of how predictable the next symbol is.
Event record
- Event date
- July 1948 – October 1948
- Timeline date
- Event date
- Verification
- Sources gathered automatically · September 17, 2026
- Lines
- ID
- evt-0064
The paper appeared in two parts, July and October 1948.
Related events
Later
- Contrasts IRE Symposium on Information Theory 1956links to this record
Chomsky's paper directly denies that the Markov source models taken from information theory are sufficient.
- Builds on Statistical speech recognition 1976links to this record
Decoding is posed as a noisy channel problem from communication theory.
- Builds on Statistical machine translation 1988links to this record
The noisy channel model is taken directly from communication theory.
- Builds on A texture grown from a sample 1999links to this record
The paper derives its method from Shannon: his n-grams of 1948, which produce English-looking text, carried into two dimensions.