The unreasonable effectiveness of data
Three Google researchers argued that for language a simpler model on enormous data beats an elegant model on little, so effort belongs in the data.
Why it matters
A position the industry had reached empirically was finally stated as a principle, and the field began measuring itself by corpus size.
The title echoes Wigner's essay on the unreasonable effectiveness of mathematics. The authors insist that language has no short description of the kind physics has, so it must be modelled by data rather than theory. The thesis proved both right and one-sided: scale gave a great deal, but the move from n-grams to transformers was a change of model.