Deepfake Video Detection with CoAtNet Variations

Deepfakes are synthesised images, videos, audio or text generated using advanced artificial intelligence. They are often so realistic that humans struggle to determine their authenticity. Facial deepfakes pose significant societal risks, including disinformation, identity fraud, reputational damage, and biometric authentication attacks, making them one of the most important and widely studied forms of synthetic media. Despite extensive research, key challenges remain, most notably generalisation, computational cost, and temporal aggregation.
This thesis provides an extensive exploration of CoAtNet architecture variants for deepfake video detection, motivated by CoAtNet's ability to integrate convolutional inductive biases with attention-based global feature learning, examining generalisation and efficiency across frame- and video-level settings and in both within- and cross-dataset regimes. We first quantified the baseline CoAtNet performance using implicit features and then introduced explicit, landmark-driven representations to test whether facial patch cues can sustain performance while enabling more efficient models. The CoAtNet model processes single frames (images), but the results from multiple input frames can be aggregated. We also propose and evaluate a 3D-CoAtNet model that processes multiple frames simultaneously.
As contributions, the thesis provides a comprehensive CoAtNet framework that assesses generalisation for deepfake video detection, proposes an enhanced variant (CoAtNet16A) for improved cross-dataset performance, and systematically studies frame-selection strategies (middle, random, and optical-flow frames). We further introduce PatchCoAtNet, which replaces full-face inputs with images composed of landmark-based image ‘patches’ and uses ensemble learning to retain the detection performance at substantially lower input costs. Finally, we present 3D-CoAtNet, which inflates CoAtNet’s 2D convolutions, pooling, residual, and attention layers into a spatiotemporal (3D) architecture to provide an alternative to multiple frame results aggregation and to provide a means of modelling dependencies across consecutive frames for video-level detections. Collectively, these contributions advance the field by improving generalisation, efficiency and temporal modelling.