Atlas record: https://www.aitimeline.dev/en/events/evt-0213/
Research · June 2009
Deep networks on graphics processors
Raina, Madhavan and Ng showed that a model with a hundred million parameters trains on graphics processors in a day instead of several weeks.
Why it matters
The computational obstacle that kept deep networks small disappeared, and that opened the road to 2012.
The speedup was roughly seventyfold against a central processor. The authors state plainly that model scale is from now on limited by the hardware budget rather than by the algorithm. This is the link between CUDA in 2007 and AlexNet in 2012: first it was shown to be possible, then it was shown to win.
Event record
- Event date
- June 2009
- Timeline date
- Event date
- Verification
- Sources gathered automatically · September 17, 2026
- Lines
- ID
- evt-0213
Presented at ICML in June 2009.
Related events
Earlier
Later
- Related AlexNet at ImageNet 2012
The recipe AlexNet was trained by three years later.
- Related The Google network that found cats 2012links to this record
The same scaling recipe, on thousands of machines instead of graphics processors.
- Related Theano 2012links to this record
An expression compiler that moves the computation onto a graphics processor by itself.
- Related Baidu's Institute of Deep Learning 2013links to this record
The bet is on the same recipe: deep networks on graphics processors.
- Related TensorFlow 2015links to this record
A tool that makes distributed training on accelerators available outside large companies.
- Related NVIDIA tensor cores 2017links to this record
The same line of accelerating training, taken to dedicated matrix units.
- Related NVIDIA becomes the most valuable company in the world 2024links to this record
The company whose gaming cards accidentally became the tool for training networks ended up at the top of the market.