LSTM
LSTM was proposed for sequence dependencies.
Why it matters
LSTM addressed the difficulty of retaining information over long sequences.
Its result concerns a recurrent architecture, not every later language model.
Atlas record: https://www.aitimeline.dev/en/events/evt-0005/
Research · November 1997
Placed by the contemporary primary publication. The exact event date is not known; its documented interval appears below.
Event record
- Event date
- 1997
- Timeline date
- Primary publication date
- Verification
- Sources gathered automatically · September 11, 2026
- Lines
- ID
- evt-0005
Sources
- primary Long Short-Term Memory
Neural Computation · Published November 1997
- primary Long Short-Term Memory, Technical Report FKI-207-95, revised December 1996 (Hochreiter, Schmidhuber)
Institute for Machine Learning, Johannes Kepler University Linz (author archive) · Published December 1996
Related events
Earlier
- Related Why long-term dependencies are not learned 1994links to this record
With Hochreiter's work it justifies the need for a new recurrent scheme.
- Related The vanishing gradient problem 1991links to this record
LSTM in 1997 is the direct answer to this diagnosis.
- Related Finding Structure in Time 1990links to this record
The recurrent scheme whose limitation LSTM later solves.
- Related Back-propagation 1986
A recurrent architecture is trained by the gradient method popularised earlier.
Later
- Builds on The gated recurrent unit (GRU) 2014links to this record
The preprint says the new hidden unit was motivated by the LSTM unit but is much simpler.
- Builds on Sequence-to-sequence 2014links to this record
The authors explicitly use multilayer LSTM networks to encode the input sequence and generate a translation.
- Sequence to Sequence Learning · arXiv
- Related Mamba 2023links to this record
The model belongs to the line of recurrent schemes with a controlled state.