Shape from Shading under Relaxed Assumptions

Shape-from-shading is a classical problem in computer vision. The aim is to recover 3D surface
shape from a single image of an object, based on a photometric analysis of the pattern of shading.
Since the amount of light reflected by a surface is a function of the direction of the incident light
and the viewer, relative to the local surface normal, image intensity conveys information about
surface orientation and hence, indirectly, surface height. This thesis aims to relax some of the
assumptions typically made in the shape-from-shading literature. Our aim is to move shape-fromshading
away from the overly simplistic assumptions which have limited its applicability to images
captured in lab conditions. In particular, we focus on assumptions of surface smoothness, point
source illumination and reconstruction in the surface normal domain.
In Chapter 3, we relax the assumption of a smooth surface. We exploit a psychology-inspired
heuristic that pixels need only have a similar surface orientation if they are both in close proximity
and have a similar intensity. This leads to an adaptive smoothing process which is able to preserve
fine surface structure. We adapt a geometric shape-from-shading framework to overcome
the problem of normals “flipping” between solutions which alternately satisfy data-closeness and
smoothness terms. Under the classical assumption of point source illumination, we show that our
method significantly outperforms a number of previously reported methods.
In Chapter 4, we relax the assumption of illumination being provided by a single point light
source. Specifically, we consider environment illumination in which lighting is represented by
a spherical function which describes the incident radiance from all directions in the scene. We
use the well known result that Lambertian reflectance acts like a low pass filter and hence the
convolution of environment lighting and surface reflectance can be efficiently represented using a
low order spherical harmonic. With an order-1 approximation, we show how the image irradiance
equation can be solved as a quadratically-constrained linear least squares optimisation. The global
optimum is found using the method of Lagrange multipliers. The order-2 case is non-convex and
prone to converge on local minima if solved using local optimisation. We reformulate the problem
as a bilinear system of equations which leads to an efficient and robust solution method. In both cases, we incorporate a structure-preserving smoothness constraint based on ideas from Chapter 3
to regularise the problem.
In Chapter 5, we continue with the relaxed illumination assumption (i.e. we model environment
illumination), but we develop algorithms which operate in the domain of surface height rather
than surface normals. This has the advantage of reducing the dimensionality of the problem at the
expense of increased complexity. Moreover, integrability is implicitly enforced via the problem
formulation. We describe two contributions. The first is a linear method for recovering surface
height directly from images formed by taking ratios between colour channels. In this case, the
nonlinear normalisation term is factored out. This allows us to form a linear system of equations
relating image intensity and surface height via a finite difference approximation to the surface gradient.
Finally, we relax the assumption that the object must be globally convex (i.e. contains no
self occlusions). We show that self occluded intensity can be related to unoccluded intensity via a
quadratic inequality constraint. This is too weak a constraint to be used for shape-from-shading on
its own. However, we use it to develop an occlusion-sensitive surface integration algorithm. We
show that the problem can be formulated as a convex optimisation and solved using semidefinite
programming.