ANNA: a convolutional network on an analog chip
In 1991 Bernhard Boser, Yann LeCun and colleagues at AT&T Bell Labs described ANNA, a chip that performs over 2,000 multiplications and additions at once. It ran a convolutional network for handwritten digits with 133,000 connections: more than 1,000 characters a second at 5.3% error against 4.9% for the original floating-point network.
Why it matters
A convolutional network that a few years earlier read postal codes on an ordinary computer went onto a special-purpose chip with little loss of accuracy, although its weights had only 6 bits. It is an early measurement of how much precision a network actually needs, a question integer accelerators faced again a quarter of a century later.
By the paper in the IEEE Journal of Solid-State Circuits (December 1991): 4,096 physical weights of 6-bit accuracy, neuron states of 3 bits, a 20 MHz clock, a sustained 5 billion connections a second, over 180,000 transistors in 0.9-micron CMOS. Weights are held as charge on capacitors and refreshed periodically from an external memory; learning is done off the chip, on a floating-point workstation. The network has five layers: the first four, convolutional, with 97% of the connections, run on the chip, the last, 3,000 connections, on a DSP32C signal processor. Simple quantisation of the weights to 6 bits lost unacceptable accuracy, so the last layer was retrained on data taken from the chip itself. Speed is over 1,000 characters a second, a hundred times a DSP implementation; a person errs on 2.5% of the same data. What the record does not claim: the date of the ISSCC 1991 paper Crossref also lists; it was not read.