Hopfield networks

Hopfield showed that a network with symmetric feedback connections has an energy that only decreases, so the system settles into stored states.

The network works as an associative memory: given part of a pattern it restores the whole. Convergence is shown by a direct calculation: the energy E, minus one half of the sum of T_ij V_i V_j, never increases when a neuron is updated, so changes continue until a local minimum. The paper notes that this case is isomorphic to an Ising model and that a symmetric random T_ij is a spin glass; it does not use the word Lyapunov. In Hopfield's own simulations about 0.15 of the number of units could be remembered before recall became badly faulty. The paper appeared in years when few were interested in networks, and it is what made the subject respectable again; Hopfield received the 2024 Nobel Prize in Physics for this line of work.