Research · July 2006
Geoffrey Hinton and co-authors described how to train deep networks layer by layer before fine-tuning them.
The publication brought attention back to deep architectures at a time when they were considered hard to train.
The initialisation scheme was later largely displaced by other methods; the historical weight of the paper is in restoring interest in depth.
Neural Computation · Published July 2006
Both concern training deep networks; this one adds layer-by-layer initialisation before fine-tuning.
Both belong to the period when model quality began to be compared on shared open data.
The same deep belief networks, two orders of magnitude larger.
Large-scale deep unsupervised learning using graphics processorsThe acoustic model is built on deep belief networks.
Deep Belief Networks for phone recognition