The widespread accessibility of multimedia content through social platforms has significantly increased the risk of image manipulation, thereby amplifying the dissemination and influence of fabricated visual information. Addressing this critical challenge necessitates the development of robust, efficient, and explainable methods to verify the authenticity of visual content. This thesis investigates novel AI-based approaches for image manipulation detection, with a particular emphasis on achieving strong generalization across challenging datasets and enhancing model interpretability.
Specifically, three deep learning architectures are proposed to address different tasks within image manipulation detection and localization: 1) WCBnet introduces an adaptive cross-block weighting mechanism at the convolutional block level, allowing the network to dynamically fuse low-level and high-level features based on their relevance to manipulation cues. This hierarchical feature weighting strategy enables WCBnet to achieve fast and accurate fact-checking with minimal additional computational overhead (only a 2.3% increase in trainable parameters), making it particularly suitable for large-scale scenarios involving newly generated manipulated images. 2) DenseWCBnet extends WCBnet by integrating multi-scale receptive fields across multiple convolutional blocks. Through adaptive fusion across different spatial dimensions, DenseWCBnet generates densely weighted feature representations that further enhance manipulation detection accuracy and significantly improve robustness against diverse and challenging manipulation types. 3) WSWCBnet proposes a novel weakly supervised localization framework, combining image-level manipulation heatmaps and semantically irrelevant segmented maps to localize manipulated regions without the need for pixel-level annotations. By leveraging only image-level supervision, WSWCBnet achieves pixel-level localization performance comparable to fully supervised methods, substantially reducing the annotation burden while maintaining high localization precision.
Extensive experiments demonstrate that WCBnet and DenseWCBnet consistently outperform state-of-the-art methods across six widely-used datasets in terms of classification accuracy and F1-score. Furthermore, when evaluated on the particularly challenging DeepfakeArt generative dataset, WCBnet and DenseWCBnet achieve classification accuracies of 94% and 97%, respectively. Additionally, WSWCBnet demonstrates its effectiveness in manipulation localization, achieving comparable performance to fully supervised models despite relying solely on weak supervision. Overall, this thesis advances the field of image forensics by providing robust, explainable, and annotation-efficient deep learning solutions for image manipulation detection and localization, with strong applicability to real-world forensic scenarios.
