Verification of RoboChart Models with Neural Network Components

Current software engineering frameworks for robotics treat artificial neural networks (ANNs) components as black boxes, and existing white-box techniques consider either component-level properties, or properties involving a specific case study. A method to establish properties that may depend on all components in such a system is, as yet, undefined. Our work consists of defining such a method. First, we developed a component whose behaviour is defined by an ANN and acts as a robotic controller. Considering our application to robotics, we focus on pre-trained ANNs used for control. We define our component in the context of RoboChart, where we define modelling notation involving a meta-model and well-formedness conditions, and a process-algebraic semantics. To further support our framework, we defined an implementation of these semantics in Java and CSPM, to enable validation and discretised verification. Given these components, we then developed an approach to verify software systems involving our ANN components. This approach involves replacing existing memoryless, cyclic, controller components with ANN components, and proving that the new system does not deviate in behaviour by more than a constant ε from the original system. Moreover, we describe a strategy for automating these proofs based on Isabelle and Marabou, combining ANN-specific verification tools with general verification tools. We demonstrate our framework using a case study involving a Segway robot where we replace a PID controller with an ANN component. Our contributions can be summarised as follows: we have generated a framework that enables the modelling, validation, and verification of robotic software involving neural network components. Finally, this work represents progress towards establishing the safety and reliability of autonomous robotics.