One camera maps in real time

In October 2003 Andrew Davison presented a system in which a single moving camera at once fixes its own position and builds a map of its surroundings — at the rate frames arrive, thirty times a second. An extended Kalman filter holds the camera pose and the three-dimensional coordinates of landmarks in one state vector, and new points enter the map through a distribution over depth hypotheses.

On a 2.2 GHz Pentium, of the 33 milliseconds available per frame at 30 Hz, roughly 10 ms go to correlation searches, 5 ms to the Kalman filter update and 10 ms to the search that initialises new features. The system keeps 6 to 10 reliable features visible; the filter update time begins to grow rapidly as the number of features in the map approaches 100, beyond which other schemes would be needed. A new point starts as 100 equally weighted depth hypotheses spread over 0.5 to 5.0 metres, and becomes a landmark once that distribution is narrow enough. A recorded 20-second run is shown. The record does not claim a resolution of 320 by 240 and does not claim millimetre accuracy indoors. Neither figure is in the paper; of the structure recovery experiment it says only that the planar surfaces were accurately recovered. The paper gives the author as being with the Robotics Research Group, Department of Engineering Science, University of Oxford. The copy read sits on his later page at Imperial College London, which is what makes this easy to get wrong.