Learning deformable shape models for object tracking

The use of computer vision to locate or track objects in images has applications in a diversity of domains. It is generally recognised that the analysis of objects of interest is eased significantly by making use of models of objects. In many cases, the strongest visual feature of an object is its shape. Also, many objects of interest are non-rigid, or have a non-rigid appearance with respect to a particular viewpoint. For these reasons, there is much interest in the construction of, and tracking with, deformable shape models.

A common approach to building such a model is to apply statistics to a set of real-life training examples of an object in order to learn shape and deformation characteristics. Such methods have proved successful in many specific applications; however, they can experience inadequacies in the general case. For example, objects which exhibit non-linear deformations give rise to models which are not compact and not specific: in the process of capturing the range of valid shapes, invalid shapes also become incorporated into the model. This effect is particularly pronounced when building models from automatically-gathered training data. Also, in tracking, smooth movement and deformation is generally assumed, but is not always the case: the apparent shape of an object can change discontinuously over time due to, for example, rotations in 3D.

The work in this thesis addresses the above problems.

Two extensions to current statistical methods are described. The first makes use of polar coordinates to improve the modelling of objects which bend or pivot. The second uses a hierarchical approach to model more general complex deformations; non-linearities are broken down into smaller linear pieces in order to improve model specificity. In particular, this greatly improves the modelling of objects from automatically-gathered training data.

A new approach to tracking which complements the latter of these models is also described. Learned object shape dynamics are combined with stochastic tracking to produce a system which can track from automatically-generated models, as well as being able to handle discontinuous shape changes.

Examples are given of the use of these techniques, predominantly in the domain of hand tracking. In particular, it is shown how it is possible to track 3D objects purely from 2D models of their silhouettes.