Risk-Aware Neural Network Ensembles

Autonomous systems with safety-critical concerns such as self-driving vehicles must be able to mitigate risk by dependably detecting entities that represent factors of risk in their environment (e.g., humans and obstacles). Nevertheless, the machine learning (ML) techniques that these systems use for image classification and real-time object detection disregard risk factors in their training and verification. As such, they produce ML models that place equal emphasis on the correct detection of all classes of objects of interest---including, for instance, buses, pedestrians and birds in a self-driving scenario. To address this limitation of existing solutions, this thesis proposes an approach for the development of risk-aware ML ensembles applied to image classification. The new approach (i) allows the risk of misclassification between different pairs of classes to be quantified individually, (ii) guides the training of deep neural network classifiers towards mitigating the risks that require treatment, and (iii) synthesises risk-aware ensembles with the aid of multi-objective genetic algorithms that seek to optimise the ensemble performance metrics while also mitigating risks. Additionally, the thesis extends the applicability of this approach to real-time object detection (RTOD) deep neural networks. RTOD involves detecting objects of interest and their positions within an image using bounding boxes, and the RTOD extension of the approach employs a suite of new algorithms to combine the bounding box predictions of the models from the risk-aware RTOD ensemble. Last but not least, the thesis introduces a self-adaptation approach that leverages risk-aware RTOD ensembles to improve the safety of an autonomous system. To that end, the new approach switches dynamically between ensembles with different risk-aware profiles as the system moves between regions of its operational design domain. This dynamic RTOD selection approach is shown to reduce the number of crashes and to increase the number of correct actions for a simulated autonomous vehicle.