Although much progress has been made in engineering multi-agent systems (MAS), many issues remain to be resolved. One issue is that there is a lack of techniques that can adequately evaluate the effectiveness (fault detection ability) of tests or testing techniques for MAS. Another is that there are no systematic approaches to evaluating the impact of possible semantic changes (changes in the interpretation of agent programs) on agents' behaviour and performance. This thesis introduces syntactic and semantic mutation to address these two issues.
Syntactic mutation is a technique that systematically generates variants ("syntactic mutants") of a description (usually a program) following a set of rules ("syntactic mutation operators"). Each mutant is expected to simulate a real description fault, therefore, the effectiveness of a test set can be evaluated by checking whether it can detect each simulated fault, in other words, distinguish the original description from each mutant. Although syntactic mutation is widely considered very effective, only limited work has been done to introduce it into MAS. This thesis extends syntactic mutation for MAS by proposing a set of syntactic mutation operators for the Jason agent language and showing that they can be used to generate real faults in Jason agent programs.
By contrast, semantic mutation systematically generates variant interpretations ("semantic mutants") of a description following a set of rules ("semantic mutation operators"). Semantic mutation has two uses: to evaluate the effectiveness of a test set by simulating faults caused by misunderstandings of how the description is interpreted, and to evaluate the impact of possible semantic changes on agents' behaviour and performance. This thesis, for the first time, proposes semantic mutation for MAS, more specifically, for three logic based agent languages, namely Jason, GOAL and 2APL. It proposes semantic mutation operators for these languages, shows that the operators for Jason can represent real misunderstandings and are practically useful.
