Werbos's thesis on computing derivatives backwards
Paul Werbos presented a Harvard thesis showing how to compute all the derivatives of a complex model's error in one backward pass down an ordered table of operations. He called the method dynamic feedback.
Why it matters
It shows the backpropagation idea was formulated well before it became widely used in 1986.
By its title page it is a thesis for a PhD in statistics, presented to Harvard's Committee on Applied Mathematics in August 1974. The model's error is computed forwards and the derivatives backwards, from the top of the table to the bottom, and the method serves any ordered table of differentiable operations. It is applied to statistical models: Werbos wrote in 1990 that the first practical application was a dynamic model to predict nationalism and social communications. The thesis mentions neural networks only as a hypothesis about the brain. In his 1990 article Werbos dates the thesis November 1974 and says he first presented the idea to the Harvard faculty in 1972.