Exploring machine learning applications for improving drinking water quality

Water utilities in the UK collect vast amounts of water quality data during their monitoring programs to assure that the final product that they deliver to consumers is of a high quality. This data, once checked over the compliance with the regulations, is archived and not used for further analysis. However, advanced data analytics tools, such as machine learning (ML), have the potential to uncover hidden information, regarding the complex processes that occur in the drinking water distribution systems (DWDS), from such types of data. This work contributes to the research over the application of these techniques in real world water quality problems when the water quality datasets are used as inputs. More specifically, this research investigates the potential of these techniques, by exploring their ability in analysing real drinking water quality problems and by proposing to the water utilities a new operational approach on the management of the water quality data for creating evidence that will support decision making over proactive interventions in the DWDS. The main contribution of this work is a Big Data framework that works as a guide for the water utilities to solve water quality related problems in their DWDS by applying ML applications in the data that have already been collected. This framework proposes, in the form of 4 layers, a new holistic approach that demands changes in the way the data storage, integration, analysis and visualisation is made. It also includes a novel process to facilitate the selection of the most appropriate ML technique, based on the water quality related problem and the existing data for analysis. Moreover, this research investigates the ability of some of the most common ML techniques by developing data-driven methodologies and applying them on water quality case studies for a water utility that supplies 5.5 million people. These methodologies are: a) a methodology that identifies correlations between different parameters and, thus, identifying factors that contribute in water quality deterioration; b) a methodology that predicts the risk of bacteriological deterioration in water exiting service reservoirs; c) a methodology for the short term forecast of free chlorine losses in drinking water trunk mains; d) a methodology that predicts the bacteriological behaviour of the water exiting the WTWs - flow cytometry total cell counts prediction in the WTWs outlet. The results obtained by the application of these methodologies, reported in this thesis, demonstrate the huge potential of ML techniques in both understanding the factors of deterioration and predicting future water quality behaviour. Overall, the data-driven methodologies and the framework presented in this thesis, open a new discussion to researchers regarding the identification of the appropriate data and methods for creating models that improve drinking water quality, and direct water utilities over a new data management approach to gain beneficial information for their DWDS operation and maintenance.