A Formal Theory of Inductive Inference
Solomonoff defined the probability of continuing a sequence through the length of the shortest program that generates it.
Why it matters
Prediction and compression were reduced to one quantity, the frame in which language-model scaling is described today.
The theory gives an optimal predictor that cannot be computed but sets an upper bound for every computable one. Kolmogorov complexity and minimum description length grew out of it. The claim that a better model is a shorter description of the data comes from this work.