A Formal Theory of Inductive Inference
Solomonoff defined the probability of continuing a sequence through the length of the shortest program that generates it.
Why it matters
Prediction and compression were reduced to one quantity, the frame in which language-model scaling is described today.
The theory gives a predictor that cannot be computed but that Solomonoff later calls a "gold standard" for induction: one can know how much closer one computable measure is to it than another. Kolmogorov defined his complexity independently (1965-69), and so did Chaitin (1966-69). By the author's account Kolmogorov had read the 1964 paper and Levin used it to put the a priori probability in a more exact form; Chaitin informally suggested that a short program for a sequence may be a measure of how good a theory is.