Whisper

OpenAI released a model trained on 680 thousand hours of multilingual web audio that transcribes 99 languages and translates into English.

Why it matters

Speech recognition became free, multilingual and robust to noise, and with it the line begun in 1971 was substantially closed.

The bet was on diversity of data rather than cleanliness: web recordings with imperfect subtitles instead of studio corpora. That is why the model holds up on accents, noise and background music, exactly where systems trained on Switchboard broke. Open weights meant transcription no longer required a cloud service or a subscription. Fifty-one years separate this from the ARPA programme. How many languages. The paper accepted at ICML 2023 counts 117,000 hours of non-English audio covering 96 other languages, that is 97 with English; the string 99 does not occur in it. The model card in OpenAI’s repository says 98 other languages. The 99 in the summary comes from OpenAI’s own release materials; the three counts differ in what they count, and none of them is an independent measurement.

Event record

Event date

September 21, 2022

Timeline date

Event date

Verification

Sources gathered automatically · September 22, 2026

Lines

ID

evt-0245

The model was open-sourced on 21 September 2022.

Sources

primary Introducing Whisper
OpenAI · Published September 2022
primary Robust Speech Recognition via Large-Scale Weak Supervision (Proceedings of the 40th International Conference on Machine Learning, PMLR 202)
Proceedings of Machine Learning Research · Published 2023

Related events

Earlier

Related wav2vec 2.0 2020
Two answers to the same problem: self-supervision against scale of weakly labelled data.
Related Human parity in speech recognition 2016
Instead of a record on one corpus, robustness on any recording.
Enables LibriSpeech: a thousand hours of audiobooks 2015links to this record
Whisper takes LibriSpeech as its reference dataset for robustness "due to its central role" in speech recognition, and shows that models trained on it break on other datasets.
- Librispeech: an ASR corpus based on public domain audio books (ICASSP 2015) · IEEE ICASSP 2015, authors' copy
- Robust Speech Recognition via Large-Scale Weak Supervision (Proceedings of the 40th International Conference on Machine Learning, PMLR 202) · Proceedings of Machine Learning Research
Extends Deep Speech 2014
The end-to-end scheme taken to 680 thousand hours of training data.
- Introducing Whisper · OpenAI

Later

Builds on ChatGPT sees and speaks 2023links to this record
ChatGPT's voice mode transcribes the user's speech with Whisper.
- ChatGPT can now see, hear, and speak · OpenAI
Related GPT-4o 2024links to this record
A pipeline of three models was replaced by one, and conversational latency fell to human levels.
Related Moshi 2024links to this record
Instead of recognising speech into text, the model works with audio on both sides of the conversation.
Contrasts Cohere opens a leading speech recognition model 2026links to this record
The same leaderboard puts Whisper Large v3, the earlier open model, at 7.44 per cent word error against 5.42.
- Cohere Transcribe: Open-Source Speech Recognition · Cohere