Enhancing and Evaluating Software Test Case Diversity

Diversity-based testing (DBT) has gained significant attention in software testing over the past two decades. DBT techniques use similarity metrics to exploit dissimilarity between software artefacts, such as requirements, inputs, or outputs, to address testing problems. This thesis advances the understanding and application of DBT to improve the diversity, effectiveness, and efficiency of software test suites. The research pursues three main objectives: (1) to analyse existing applications and artefacts used in DBT; (2) to develop automated approaches that apply DBT to test suite reduction and test case prioritisation; and (3) to explore new artefacts, such as bytecode, to enhance testing performance.
First, a systematic mapping study synthesised the landscape of DBT research, analysing 167 papers that employ 79 similarity metrics across 22 artefact types and 11 testing problems. The study identifies dominant trends, underexplored artefacts, and open challenges. Second, a string-based reduction technique was developed to minimise automatically generated test suites. This approach operates solely on test text, eliminating execution cost while maintaining fault detection capability. Third, the thesis investigates bytecode-level diversity as a novel artefact for test case prioritisation and mutant analysis. Results show that bytecode diversity improves fault detection by up to 7.8% and performs significantly faster than text-based approaches. Finally, the use of bytecode-based diversity in targeting stubborn mutants demonstrates superior efficiency, reducing test counts by up to 46% compared with coverage-based methods.
Overall, this thesis provides new insights into DBT theory and practice, introduces novel artefacts and techniques, and delivers scalable, effective solutions to key software testing challenges.