Atlas record: https://www.aitimeline.dev/en/events/evt-0291/
Availability · December 11, 2023
Mixtral 8x7B
Mistral released a model in which only part of the parameters work on each token, and it beat the 70-billion Llama 2 at far lower compute cost.
Why it matters
The mixture-of-experts scheme reached open release, showing that quality does not require every parameter to work every time.
The model holds 46.7 billion parameters but engages about 13 billion per token: a router picks two experts out of eight. That gives the quality of a larger model at the cost of a smaller one. The scheme was assumed to be in use inside large closed models; the open release made it checkable and reproducible.
Event record
- Event date
- December 11, 2023
- Timeline date
- Event date
- Verification
- Sources gathered automatically · September 22, 2026
- Lines
- ID
- evt-0291
The model was announced on 11 December 2023.
Related events
Earlier
- Extends Mistral 7B 2023
The same company moved to a mixture of experts two and a half months later.
- Related Llama 2 2023
Open weights without restrictions, outperforming Llama 2.
- Related The sparsely-gated mixture-of-experts layer 2017links to this record
Mixtral 8x7B works on the same sparse-gate principle: at every layer, for every token, a router picks two of eight parameter groups. Its announcement does not cite this paper.
Later
- Related Fine-grained and shared experts 2024links to this record
A month after Mixtral's open release, this work took apart why a mixture of experts works and changed how one is built.
- Related Llama 3 2024links to this record
Two open models of the same season, taking different routes to quality.
- Related DeepSeek-V3 2024links to this record
The same mixture-of-experts scheme, carried to 671 billion parameters.
- Related Llama 4 2025links to this record
The mixture-of-experts scheme first opened in 2023 became the basis of the whole family here.