WaveNet

DeepMind trained a network to generate audio sample by sample, and in blind tests synthesised speech became more natural than the best existing systems, with the gap to human speech more than halved.

Why it matters

Speech synthesis moved from stitched-together recorded fragments and parametric models to a generative model of raw audio.

Existing systems either assembled speech from recorded pieces or generated it parametrically, and the parametric ones sounded less natural than the concatenative. WaveNet models the distribution of each next sample given all previous ones, using dilated causal convolutions. In the English naturalness test WaveNet scored 4.21 out of 5 against 4.55 for natural speech: the gap to the best synthetic system fell from 0.69 to 0.34 but did not close. The first version was too slow for practical use; an optimised version was running in Google Assistant a year later.

Event record

Event date

September 2016

Timeline date

Event date

Verification

Sources gathered automatically · September 17, 2026

Lines

ID

evt-0232

Announcement of 8 September 2016.

Sources

primary WaveNet: A generative model for raw audio
DeepMind · Published September 2016
primary WaveNet: A Generative Model for Raw Audio
arXiv (van den Oord and others, Google DeepMind and Google) · Published September 2016
primary WaveNet launches in the Google Assistant
Google DeepMind · Published October 4, 2017

Related events

Earlier

Builds on AlexNet at ImageNet 2012
The model is built on convolutions, causal and dilated ones.
- WaveNet: A generative model for raw audio · DeepMind
Enables A voice from a statistical model: HTS 1999links to this record
WaveNet describes statistical parametric synthesis with HMMs, citing Yoshimura's 2002 thesis, and takes a parametric system as a baseline.
- WaveNet: A Generative Model for Raw Audio (A. van den Oord, S. Dieleman, H. Zen et al.), arXiv:1609.03499v1, 12 September 2016 · arXiv (Google DeepMind)
Enables Unit selection: a voice from pieces of recording 1996links to this record
WaveNet names this paper among the foundations of concatenative synthesis and takes a unit selection system as its baseline.
- WaveNet: A Generative Model for Raw Audio (A. van den Oord, S. Dieleman, H. Zen et al.), arXiv:1609.03499v1, 12 September 2016 · arXiv (Google DeepMind)
Related Dynamic time warping 1978
Sound is generated sample by sample instead of through a coefficient filter.

Later

Contrasts Tacotron: speech straight from letters 2017links to this record
The authors set themselves against WaveNet: slow because it generates sample by sample, and not end to end because it needs linguistic features from a separate frontend; Tacotron generates by frame and learns from text-audio pairs.
- Tacotron: A Fully End-to-End Text-to-Speech Synthesis Model (arXiv:1703.10135v1) · arXiv (Google)
Builds on Jukebox: songs with singing as raw audio 2020links to this record
Jukebox's VQ-VAE encoders and decoders are built from "WaveNet-style" residual networks; the paper names WaveNet as the forerunner of autoregressive raw-audio generation.
- Jukebox: A Generative Model for Music (arXiv:2005.00341v1) · arXiv (OpenAI)
Related GPT-4o 2024links to this record
Generating sound directly, without an intermediate text stage.
Related Moshi 2024links to this record
Generating audio directly with a neural network, eight years on.