TD-Gammon
Tesauro's program learned to play backgammon from scratch purely by playing itself, and reached a fairly strong intermediate level, stronger than commercial programs and than a network trained on a large database of expert moves.
Why it matters
The author calls it apparently the first application of temporal-difference learning to a complex non-trivial task, and the result was better than the theory of the day predicted.
The network evaluated a position and temporal-difference learning corrected that estimate against the next one. The best network (40 hidden units) trained for about 200,000 games: it won 66.2 per cent against Gammontool (the network trained on expert examples, 59.4 per cent) and played 50-50 against Neurogammon 1.0, champion of the 1989 Computer Olympiad. The level of the best humans belongs to later versions of TD-Gammon, not to this paper: by Tesauro's 1995 article, version 1.0 played 51 games against grandmasters at the end of 1991 and lost 13 points, and an analysis of the program published in Robertie's 1992 article showed that the near-universal opening move (the slot) was inferior to splitting the back checkers. This is the same recipe that produced AlphaGo twenty-four years later.