Availability · February 24, 2023

LLaMA

Meta released a family of models from 7 to 65 billion parameters, trained only on public data, and gave the weights to researchers.

Why it matters

A smaller model beat one ten times its size, and weights that leaked within a week spawned a whole open-model industry.

Weights were issued on application, but within a week they appeared in the open without Meta's permission. The company did not pursue the distribution, and Alpaca, Vicuna and hundreds of fine-tuned variants grew on LLaMA within months. That broke the assumption that frontier models would stay inside a few companies: fine-tuning someone else's base turned out to cost hundreds of dollars.

Event record

Event date

February 24, 2023

Timeline date

Event date

Verification

Sources gathered automatically · September 17, 2026

Lines

Openness

ID

evt-0271

The models were announced on 24 February 2023; the preprint is dated 27 February.

Sources

primary LLaMA: Open and Efficient Foundation Language Models
arXiv · Published February 2023
secondary Llama (language model)
Wikipedia

Related events

Related GPT-3
A 13-billion-parameter model beats the 175-billion GPT-3 on most measures.
Related Stable Diffusion
The second event of the period to put a capable model outside its developer's walls.

Records that link to this one

Builds on Alpaca
The model is a fine-tune of the 7-billion-parameter LLaMA.
Alpaca: A Strong, Replicable Instruction-Following Model
Related QLoRA
The method makes fine-tuning open models possible on a single graphics card.
Extends Llama 2
The weights are now released openly and commercial use is permitted.
Meta and Microsoft Introduce the Next Generation of Llama
Builds on Three per cent of a training budget, spent on one language
Less than two months after the LLaMA weights appeared, a Brazilian group further pretrained them in Portuguese.
Sabia: Portuguese Large Language Models (arXiv:2304.07880v1)
Contrasts Open weights out of Abu Dhabi
Three months later weights of the same class went out not under a research licence but royalty-free, and from Abu Dhabi rather than California.
Falcon LLM, Technology Innovation Institute (archived 8 June 2023)
Enables Rotary position embedding (RoPE)
LLaMA removes the absolute positional embeddings and adds rotary positional embeddings (RoPE), 'introduced by Su et al. (2021)', at each layer.
LLaMA: Open and Efficient Foundation Language Models
Enables Chinchilla: compute-optimal training
LLaMA cites the recommendation of Hoffmann et al. (2022) to train a 10B model on 200B tokens and departs from it: a 7B model keeps improving even after a trillion tokens.
LLaMA: Open and Efficient Foundation Language Models