A model-driven engineering approach for monitoring ML model performance

With a rising number of enterprises adopting machine learning (ML) in their operations, the issue of ML monitoring to ensure robustness has become increasingly relevant. Unfortunately, implementing ML monitoring systems has proven challenging partly because it requires cross-discipline collaboration between data scientists and software engineers. This thesis hypothesises that a solution centred around model-driven engineering (MDE) comprising a domain-specific language and an accompanying execution environment can address many of the challenges associated with ML monitoring. To evaluate the validity of this hypothesis, such a solution was designed at the architectural level and implemented. The solution’s design offers portability, extensibility and separation of concerns between data scientists and software engineers. This is validated through empirical studies involving professional data scientists. In addition, three case studies with third-party ML models have been developed to further evaluate the solution’s validity.