Immersive audio and network music performance: practical applications and user testing

Network Music Performance (NMP) technology allows remote musicians to perform together in real-time over Wide Area Networks (WANs) using low-latency audio streaming methods. Musicians hear and interact with one another, experiencing musical timing as if performing together live. The extension of NMP systems with immersive audio (Immersive Network Music Performance, INMP) allows for simulation of hearing networked performance as if cohabiting a physical 3-Dimensional (3-D) acoustic environment with remote performers. This provides an enhancement of traditional NMP systems, and provides Audio-Visual (AV) coherence in navigation of 3-D immersive display and interface using technologies such as Virtual and Extended Reality (VR/XR).

This thesis proposes that low-latency INMP systems are viable as a practical remote performance workflow, and that immersive audio will improve the User Experience (UX). A series of studies contribute documentation of low-latency INMP system designs which have been implemented in practical use-case contexts, practice-oriented user-testing of these designs, and investigation of the experience of immersive audio.

The first three studies contribute case-studies of INMP systems in the use-cases of remote education, remote virtual production, and remote concert. Results demonstrate the viability of these solutions for supporting remote synchronous performance and positive performance experience with minimal prior training. UX investigation demonstrates the potential for experience improvement and deterioration associated with immersive audio, and highlights relevant practical challenges, informing recommendations for INMP experience optimisation. Novel demonstration of performer engagement with video and VR/XR systems is provided, and limitations of visual display and interface methods are highlighted.

A final study contributes testing of the influence of latency and acoustic parameters in INMP for vocal performance. Results provide novel demonstration of variation in INMP experience quality for different Reverb Times. Here findings inform recommendations for INMP experience optimisation.