Automated Testing of User Interactions in Extended Reality Applications

Extended Reality (XR) is a rapidly emerging technology that offers immersive user experiences across diverse application domains. Unlike traditional two-dimensional graphical user interfaces, XR features interactions with six degrees of freedom, introducing new paradigms for user interaction and posing unique software engineering challenges. Among these, the validation of XR software through effective testing stands as a crucial problem requiring innovative solutions.

Current research in XR software testing has produced approaches that support scene navigation and basic interaction activation. However, there remains a significant gap: existing methods cannot automatically generate or execute realistic spatial user inputs, such as grab and trigger actions via hand-held controllers or hand gestures. Furthermore, present metrics do not robustly capture the diverse coverage of these spatial interactions, and the field still lacks a systematic understanding of key testing methodologies and concerns specific to XR environments.

This thesis addresses these challenges with two main objectives: (1) to investigate the landscape of XR software testing and establish foundational knowledge for XR-specific user interaction testing, and (2) to develop and evaluate new techniques that systematically generate and validate realistic user interactions in XR environments. To achieve these goals, five major contributions are presented: (1) the first systematic mapping study of XR software testing, based on an analysis of 34 published papers; (2) a novel test automation library designed around a new taxonomy of XR user interactions; (3) an automated technique for generating and executing XR user interactions, informed by a categorisation from open-source XR projects; (4) a benchmark dataset of seven XR scenes and 367 user interactions as a standard for evaluating XR testing approaches; and (5) a foundational roadmap for the future of XR devices, software, and testing, outlining structured research challenges and mitigation strategies. Collectively, these contributions advance systematic and scalable testing for XR applications, supporting both future research and practical development in this fast-evolving field.