What Can Computational Models Tell Us About Scene Memorability?

Computational memorability prediction has allowed significant advances in the understanding of human visual memory; and in turn, advances in understanding what makes an image memorable. Recently, this research has expanded to the second dimension, with Visual Memory Schemas (VMS) maps revealing the specific regions in a scene that lead to that scene being remembered. In this thesis, we explore the concept of VMS maps in detail, develop new VMS datasets, novel models for VMS prediction, explore whether human memory can be modulated with VMS maps, and finally investigate the relationship between scene memorability and scene complexity. We propose three new approaches for predicting visual memory schemas, starting with a variational autoencoder-based model, before exploring the role of self-attention, multi-scale information, and depth in the prediction of scene memorability. Based upon this work, we develop a novel "dual-feedback" model that uses both VMS datasets and pre-existing single-score memorability datasets to predict memorability maps for scene images, setting a new state-of-the-art for VMS prediction. This work is supported by our efforts in expanding VMS datasets; from the original 800 images, up to a dataset of over 4000+ scenes and VMS maps. We make use of our VMS predictors by integrating them with generative models with the goal of synthesising scene images of controllable memorability. We test our generated scenes against real-world human observers and find that images we synthesise to be more memorable have a greater hit-rate than images we synthesise to be less memorable. Finally, we investigate the relationship between scene complexity and scene memorability, developing novel techniques and architectures capable of predicting how complex a human finds a scene, and ultimately finding that the complexity of the scene plays a small, but significant role, in the memorability of that scene.