A doubling every 2.46 years, not every year
On 27 June 2022 Epoch AI measured the rate at which compute gets cheaper across 470 models of graphics card released from 2006 to 2021: FLOP/s per dollar doubles every 2.46 years. Huang's law - a 25-fold gain every five years, a doubling every 1.08 years - does not fit the data.
Why it matters
Until this, the rate at which hardware gets cheaper was either a marketing claim or an estimate on a small sample, with prior figures scattered from 1.5 to 4.4 years. Here is a number with a confidence interval, and it is slower than Moore's law. Every later calculation of what training costs either rests on this figure or argues with it.
The results from the report's table, with 95 per cent intervals: the whole dataset at 2.46 years [2.24, 2.72], which is 0.122 orders of magnitude a year. The cards actually used in machine learning research, 26 of them, at 2.07 years [1.54, 3.13]. The best cards by FLOP/s per dollar in each month, 57 of them, at 2.95 years [2.54, 3.52]. Half precision, 91 cards, at 2.30 years. For comparison the report puts Moore's law at 2 years and Huang's law at 1.08, taken from Jensen Huang's own "25x improvement every 5 years". Two cautions a retelling usually drops. The faster figure for machine-learning cards is not statistically significantly different from the whole dataset, while the slower figure for top cards is. And the authors offer their explanation of it as a possibility rather than a finding: it "could be explained by relevant labs spending more resources on procuring top GPUs over time", that is, by procurement decisions rather than by faster progress in the hardware itself. The report's own conclusion is about 2.5 years. The research report this record was made from gave as a second source a preprint titled "Compute Trends Across Three Eras of Machine Learning" at arXiv:2311.09227. That number holds an entirely different paper - "Open-Sourcing Highly Capable Foundation Models" by Elizabeth Seger and twenty-one other authors, submitted on 29 September 2023. The source was dropped.