Tracking and mapping in separate threads
In November 2007 Georg Klein and David Murray presented PTAM: tracking the camera and building the map stopped being one computation and moved apart into two parallel threads on a dual-core machine. The tracking thread keeps up with every frame while the mapping thread runs batch optimisation over keyframes — and the map grows to thousands of points.
Why it matters
Map building left the real-time constraint without leaving the real-time system. Every notable visual SLAM system since has repeated this split into two threads, and position tracking in current augmented reality applications rests on it.
The map in the examples shown holds 2000 to 6000 points and 40 to 120 keyframes — where the filter-based systems of the time held on at a hundred landmarks. Frames of 640 by 480 arrive from a Unibrain Fire-i camera at 30 Hz. In the snapshot opening the paper the map holds close to 3000 features, the system attempted to find 1000 of them in the current frame, found 660, and processed that frame in 18 ms. Tracking on a map of 4000 points takes around 20 ms per frame. Twice in the run it breaks: after a loss the system relocalises for several frames, which costs up to 90 ms per frame; when the camera moves far from the desk and a very large number of features appear, a frame costs around 30 ms. The record does not claim "sub-millimetre positioning error". No such figure is in the paper.