Detection, tracking and event analysis are areas of video analysis which have great importance in robotics applications and automated surveillance. Although they have been greatly studied individually, there has been little work on performing them jointly where they mutually influence and improve each other. In this thesis we present a novel approach for jointly estimating the track of a moving object and recognising the events in which it participates.
The contributions are divided into three main chapters. In the first, we will introduce our geometric carried object detector which allows to detect a generic class of objects. This detector primarily uses geometric shape models instead of using pre-trained object class models and does not solely rely on protrusion regions.
The second main chapter presents our spatial consistency tracker which incorporates events at a detection level within a tracklet building process. This tracker enforces spatial consistency between objects and other pre-tracked entities in the scene.
Finally, in the third main chapter we present our joint tracking and event analysis framework posed as maximisation of a posterior probability defined over event sequences and temporally-disjoint subsets of tracklets. In this framework events are incorporated at a tracking level, where tracking and event analysis mutually influence and improve each other.
We evaluate the aforementioned framework using three datasets. We compare our detector and spatial consistency tracker against a state-of-the-art detector by providing detection and tracking results. We evaluate the tracking performance of our joint tracking and event analysis framework using tracklets from two state of the art trackers, and additionally our own from our spatial consistency tracker; we demonstrate improved tracking performance in each case due to jointly incorporating events within the tracking process, while also subsequently improving event recognition.
