The TREC video track: a shared test for video search
At TREC 2001 in Gaithersburg (13-16 November 2001) NIST ran its first video track: 12 groups from the US, Asia and Europe searched 11 hours of MPEG-1 video for 74 topics, each carrying a clip, image or sound besides its text, and detected shot boundaries. From 2003 the track became TRECVID, a separate annual evaluation.
Why it matters
Video search gained what text retrieval had had since 1992: a shared collection, the same queries and the same measures, so that systems from different laboratories could be compared for the first time. The query itself was multimodal here, and speech transcripts were only one permitted source that was not meant to stand in for analysing the picture.
The video came from the Open Video Project of the University of North Carolina, the NIST Digital Video Library and BBC stock shots. The shot boundary task used 42 videos, about 5.8 hours and 3.34 GB; the transitions, cuts, dissolves and fades, were marked by a NIST student. Transcripts from automatic speech recognition were allowed provided the group also submitted a run without them. The report by Smeaton (Dublin City University), Over and Taban (NIST) is dated 18 April 2002; it calls the track's first year "a bit unorthodox" by the standards of mature TREC tracks. What this record does not assert: the report carries no results; they are in the conference tables, which were not opened. The TREC 2001 overview was not read: its PDF has no usable text layer.