Exploring the Psychophysiological Influence of Audio Presentation on Film Viewing Experiences

Audio presentation is a holistic term encapsulating all aspects of how sound is designed and delivered to an audience. In the context of film viewing, audio presentation is theorised to be profoundly influential — capable of greatly enhancing or diminishing positive experiential outcomes. This thesis examines this influence, focusing on the relationship between multichannel soundtracks and “immersion” through the use of psychophysiological methods. Such methods, which use bodily measures to infer mental states, have seldom been used in this context.

The empirically supported results of this thesis suggest that variations in domestic multichannel audio systems have a less significant effect on the experience of viewing a film than commonly theorised. This result is primarily discussed in terms of two conflicting aims associated with media creation. On one hand, content creators wish to deliver experiences that utilise the best available reproduction technologies. On the other, they aim to ensure that their artistic vision translates to an ever-growing ecosystem of playback devices and scenarios.

Another outcome of this project was the development of a novel method which utilises a discreetly placed camera to obtain psychologically insightful physiological features. This method is highlighted as a significant research contribution. Notably, the conceptualised camera-sourced intersubject heart rate correlation feature (ISC-HR) was found to be both accurate, compared to ground truth, and sensitive, compared to the target variable of “immersion”. Moreover, the camera-sourced facial action and gaze location ISC features explored appear to be promising additional means of evaluating audiovisual experiences.

A dedicated study was also conducted as part of this project, considering the robustness of the camera-based method, evaluating the camera-sourced data obtained under dark room conditions as well as two brighter illumination conditions. This led to the finding that bright and stable illumination is imperative for obtaining reliable data and helped identify specific characteristic errors which require consideration when using this camera-based approach.