Finding Structure in Time
Elman gave a network a context layer holding its own previous state, and showed that it then learns the structure of a sequence.
Why it matters
Time stopped being an extra input and became internal state, the form recurrent networks still work in.
The network predicted the next word in a sentence. Analysis of the hidden layer showed it had grouped words into nouns, verbs, and animate against inanimate by itself, though nobody had supplied categories or labels. Elman also showed the limit: the further the dependency, the weaker the signal, which is what Hochreiter explained formally the next year.