The ESP Game: image labels as play
Luis von Ahn and Laura Dabbish of Carnegie Mellon posted the ESP game on 9 August 2003: two strangers see the same image and score when they type the same word. By 10 December 13,630 players had given 1,271,451 labels for 293,760 images. The paper was presented at CHI 2004 on 25 April 2004.
Why it matters
Image labels, which computer vision could not produce and which were costly as paid work, came from a game: a word two players agreed on became a label. The ESP set later served as the yardstick ImageNet measured itself against.
A pair produced 3.89 labels a minute on average; by the authors' estimate 5,000 people playing round the clock would give one label to all 425 million images in Google in 31 days, and six words in six months. Agreed words became "taboo" for later pairs, up to six an image. Over 80% of players came back, and 33 played over 1,000 games. In a check by 15 people who had not played, 5.105 of six labels on average were judged useful and 0.105 unrelated. The record claims no link between ESP and Mechanical Turk: the paper does not mention it.