Atlas record: https://www.aitimeline.dev/en/events/evt-0262/
Availability · May 18, 2016
Google's Tensor Processing Unit
Google announced a chip of its own designed exclusively for neural networks, and said it had already been running in its data centres for a year.
Why it matters
Hardware was built for the method rather than the method fitted to existing hardware, and that changed the economics of large models.
By Google's 2017 description, the heart of the chip is a matrix multiply unit of 65,536 eight-bit elements; precision is deliberately reduced because networks tolerate it. The 2016 announcement said only an 'order of magnitude' better performance per watt, while the 2017 analysis gave 30 to 80 times against the processors and graphics accelerators of the time. The consequence runs wider than one company: specialised accelerators became an industry of their own, and access to them a condition of doing frontier research.
Event record
- Event date
- May 18, 2016
- Timeline date
- Event date
- Verification
- Sources gathered automatically · September 17, 2026
- Lines
- ID
- evt-0262
Announced on 18 May 2016; it had already been running in data centres for more than a year.
Related events
Earlier
- Related TensorFlow 2015
The accelerator and the training system come from one company and are made for each other.
- Related CUDA becomes generally available 2007
Instead of a repurposed graphics chip, a chip designed for neural networks.
- Related Neural network chips from Intel and Adaptive Solutions 1990links to this record
CNAPS and the TPU are both digital chips that multiply network weights in integers of limited width, a quarter of a century apart.
- Enables Kung and Leiserson's systolic arrays 1978links to this record
Google's 2017 TPU paper says its matrix unit uses systolic execution and cites Kung and Leiserson's 1980 chapter for it, whose first version was written in 1978.
Later
- Contrasts NVIDIA tensor cores 2017links to this record
The graphics processor maker's answer to the appearance of specialised chips.
- Builds on Google unveils its eighth-generation TPU chips 2026links to this record
The eighth generation of the same line of Google neural-network processors that began with the 2016 chip. The record states the succession of generations, not how much of the first design carried over.