Object outlines drawn by a website’s visitors
On 8 September 2005 Bryan Russell, Antonio Torralba, Kevin Murphy and William Freeman described LabelMe in an MIT memo: a web tool in which anyone outlines objects in a photo with a polygon and names them in their own words. The tool came with 10,000 images, 3000 of them already labelled; by 21 December 2006 the database held 111,490 polygons.
Why it matters
Recognition datasets had been assembled by one group for its own task, often with a few classes. LabelMe made annotation shared and open, and it recorded not only what is in a photo but where and in what shape, in scenes with many objects together. ImageNet in 2009 would call it complementary to itself: fewer categories, but with outlines.
Memo AIM-2005-025, 11 pages. The database took all the images of the MIT CSAIL database of objects and scenes and added new ones, mostly offices and streets; its section counts 455 distinct object names and 3342 images with at least one labelled object. Quality was then checked by hand: a "verified" field set by one of the authors. The journal version is International Journal of Computer Vision 77(1-3), pp. 157-173. As of 21 December 2006, of 111,490 polygons 44,059 had been drawn through the web tool and 67,431 offline, on 11,845 photos and 18,524 video frames. Names are free: online annotators entered 2888 different descriptions, and the paper merges synonyms through WordNet; users could now correct someone else’s outline themselves. The CSAIL abstracts of 2006 give a different cut: more than 77,000 labelled objects in 23,000 images and about 7500 new labels a month. The copy of the journal paper on Russell’s page carries the file name of the 2005 memo; the memo itself is in MIT DSpace.