Advancing Controllability and Explainability in Generative 3D Face Models

Three-dimensional face modelling, whether employing linear or non-linear approaches, involves mapping 3D face scans into a latent space for reconstructing 3D face shapes using this model. To enhance the interpretability of this mapped latent space for humans, a critical task emerges in computer vision with a focus on developing specific latent spaces for individual facial components rather than a single global latent space for the entire face. This thesis presents pipelines based on deep learning algorithms for the explainable and controllable non-linear modelling of 3D faces within latent spaces. Firstly, our method introduces a 3D face model that learns to map face identity and expression into two independent latent spaces, achieving face identity and expression disentanglement. This is particularly aimed at addressing limitations in scenarios lacking facial identity ground truths, which differs from other approaches. Secondly, beyond learning identity and expression latent spaces, our work further subdivides entire faces into multiple semantic regions, including the nose, eyes, mouth and others, and learns the separate latent variables for these regions through our novel framework. Additionally, we apply a Laplacian blending technique to the key facial feature swapping strategy, enhancing data augmentation and seamlessly reconstructing face shapes. Both methods are evaluated on public datasets and achieve state-of-the-art performance, demonstrating their effectiveness in reconstructing face shapes and disentangling latent variables for different facial features. The learnt latent variables are proven to be applicable to many applications, e.g. face recognition, face expression transfer and face editing. Moreover, to investigate the impact of different representations on the reconstructed face shapes, our two models employ different representations for 3D face shapes, one using explicit representations and the other employing implicit representations. Comprehensive comparative analyses are conducted to evaluate the effectiveness of our methods in 3D face modelling based on different representations and architectures.