Atlas record: https://www.aitimeline.dev/en/events/evt-0294/
Availability · February 15, 2024
A million tokens of context
Google announced Gemini 1.5 Pro with a context window of one million tokens: an hour of video, eleven hours of audio, or whole code bases in a single request.
Why it matters
The amount a model holds in view grew tens of times over, and the question of searching documents turned into the question of reading them all.
Until then a typical window was a few tens of thousands of tokens, and any work with a large corpus meant splitting it and retrieving the relevant pieces. A million tokens made that step unnecessary for most documents. In tests that hide a fact inside a long text, the model found it reliably across the whole window. That moved engineering attention away from building retrieval pipelines and towards what to do with the answer.
Event record
- Event date
- February 15, 2024
- Timeline date
- Event date
- Verification
- Sources gathered automatically · September 18, 2026
- Lines
- ID
- evt-0294
Announcement of 15 February 2024.
Related events
Earlier
- Extends Gemini 1.0 2023
The next generation of the same family, announced two months later.
- Related Transformer 2017
The quadratic cost of attention was the main obstacle to long context.
Later
- Related Compressive memory instead of a growing KV cache 2024links to this record
Two months after a million tokens of context became available in a product, Google published a mechanism that makes such a length constant in memory.
- Extends Gemini 2.0 Flash 2024links to this record
The next generation of the same family, ten months later.