The Assurance of Bayesian Networks for Mission Critical Systems

A prerequisite for the assurance of any mission-critical system is a comprehensive understanding of a system’s properties and behaviours. This is a challenging proposition for many AI-based Systems (AISs). Their functionality is often dictated by factors that are often outside the scope of the assurance concerns typical of conventional software systems. These distinctions have implications for all phases of the design, development, deployment and operation of AISs. They pose serious problems for existing software assurance standards, guidelines and techniques: the application of existing practices to an AIS will fail to expose or mitigate numerous system aspects that can contribute to hazardous system behaviours.
This thesis introduces a number of techniques that aim to support the resolution of these problems for Bayesian Network-based Systems (BNSs). This class of system has been deployed in many applications, ranging from medical diagnostic systems to naviga- tional controls aboard autonomous systems. To date, there is no published literature on the deployment of these systems in directly safety-critical roles. This thesis introduces ap- proaches aimed at addressing three particular challenges. Firstly, it proposes a framework for conceptualising and communicating the distinctions between BNSs and conventional software systems and uses this framework to generate and refine a set of BNS verification and validation objectives. Secondly, it introduces an assurance-focussed BNS analysis technique that can provide targeted information on mission-critical aspects of a BNS. Finally, it outlines an approach for describing how BNS-specific safety evidence relates to BNS aspects, and how the evidence can be used to derive sufficient confidence in a mission-critical BNS.
These contributions are then evaluated in the context of a case study that indicates the utility of the proposed techniques, and how these can be used to comprehensively structure and target the unconventional assurance concerns associated with the development of a mission-critical BNS.