The end of ARPA SUR: Harpy against the 1971 goals

In September 1976, with four systems demonstrated, the five-year ARPA speech understanding programme ended. The report of Carnegie-Mellon University to DARPA sets Harpy beside the goals of November 1971 — 1000 words, under 10 % semantic error, about 300 MIPSS: Harpy had 1011 words, 5 % error and 28 MIPSS. Klatt of MIT calls it the best of the four and says it satisfied the goals set at the start.

The goals of 1971 and Harpy's result (CMU report, first section): many speakers — five (three male, two female); a 1000-word vocabulary — 1011; under 10 % semantic error — 5 %; about 300 MIPSS — 28 MIPSS (a footnote: the original specification said «a few times real-time» on a 100 MIPS machine); 256k 36-bit words of memory; 5 dollars per sentence processed; the task is retrieval of AI abstracts. The test of 8 September 1976: Harpy — 97 % word, 91 % sentence and 95 % semantic accuracy on 184 test sentences (1138 words, 6.5 minutes of speech); 27.9 MIPSS. Hearsay-II — 86 % word, 73 % sentence, 91 % semantic accuracy, but on only 22 sentences; 85.0 MIPSS. Live on the same day Harpy recognised four of five sentences correctly, Hearsay-II three. This is the contractor's report on its own systems (rule 8). The second source is the abstract of Klatt's review (JASA, December 1977), independent of CMU: «Harpy satisfied a set of design goals that were specified at the beginning of the program», among them understanding over 90 % of sentences from a 1000-word lexicon. Klatt's article was read only in its abstract (the publisher's page is behind a bot check). What the record does not claim: the results of the other three systems (the abstract does not name them); that no goal was relaxed in the meeting of it (the report gives «5 %» and «28 MIPSS» against «10 %» and «300», but its footnote on the specification shows the 1971 goals were also worded differently); that after September 1976 DARPA cut or kept its speech funding.