Dense Optical Flow Estimation using Diffusion Distances

Diffusion maps have been shown to model relations between points by considering the overall connectivity of the graph. This report outlines how we can apply the diffusion framework to dense optical flow estimation where diffusion maps are used to embed distributions of local spatial gradients. We review the problem of dense optical flow estimation and several broad types of approaches to computing accurate estimate of the flow. We then review the diffusion framework and its predecessors in the manifold learning literature. Local image features are recorded by diffusion distances calculated from the graph Laplacian whose kernel function depends on inter-pixel intensity differences in a certain neighbourhood. These features are then used in a correlational optical flow estimation algorithm to illustrate the improvement to the dense estimate of optical flow by using a richer description of features as the elementary unit in the estimation. By considering systems of correlation vectors from image neigbourhoods, we also increase the smoothness of the estimate. The present work compares several smoothing principles, including the vector mean, vector median, marginal median which are based on both the maximum correlation and minimum rank of correlation vectors from the correlation matrix. A large number of very accurate estimates, spread through the image can be identified based on level of consensus with the estimates from surrounding pixels, which we term as confidence. We use this confidence information as a basis for smoothing the motion estimate by filling regions with poor confidence with estimates from neighboring high confidence regions. The proposed methodology was applied on two distinct image sequences from the Middlebury data set, as well as a fluid motion data set. Results show the robustness of our method to the different types of input data.