Reconstruction of extended environments from image sequences

The automatic recovery of the three-dimensional structure of a scene from a sequence of two-dimensional images has been the subject of considerable research in the field of machine vision, with applications as wide-ranging as object recognition, virtual reality and robot navigation. Traditional attempts to solve this structure from motion (SFM) problem rely on calibrated cameras and involve the detection and tracking of features through successive images in the sequence.

When considering long image sequences, taken with an ordinary hand-held video camera, the problem is significantly harder, since both camera calibration parameters and matched feature information are difficult to obtain accurately. An additional complication is that small errors in the recovered structure will accumulate over long sequences, possibly resulting in a reconstruction which is internally inconsistent. To date, there has been no discussion in the SFM literature of attempts to tackle this important issue.

Recently, a number of different techniques have been developed for scene reconstruction using uncalibrated cameras. In such cases the recovered structure is correct up to projective transformation of the real structure. In this thesis, an original, incremental reconstruction system id described, based on this calibrated approach. A novel implementation for computing the fundamental matrix from a pair of images is presented, from which a projective reconstruction is obtained. For the first image pair in sequences, a small number of ground truth points are used to upgrade from projective to Eudlidean structure. This structure is propagated through successive frames to obtain a complete Euclidean reconstruction for the entire scene. The inconsistency problem is addressed by attempting to detect when previously viewed sections of the scene are re-encountered. A solution method using the geometric hashing model-based object recognition paradigm is proposed.