Llama 3.1 at 405 billion
Meta opened the weights of a 405-billion-parameter model, the largest whose weights had ever been published, at a quality level matching frontier closed models.
Why it matters
The gap between closed and open models shrank to months, and reproducing the frontier became a question of having compute rather than permission.
The technical report described the training frankly: 15 trillion tokens, 16 thousand H100 accelerators, a list of hardware failures over the run. Nobody had published that kind of openness about frontier training engineering before. Running a 405-billion model was still within reach of few, but the existence of the weights changed the calculation: a state or a company could now hold a frontier model without asking anyone.