Statistical and Machine Learning Methods for Risk Prediction in Health

Prediction of occurrence of an event in a patients’ lifecourse is gradually becoming very important in this era of stratified medicine. With the availability of vast amounts of data in the form of Electronic Medical Records (EMRs), many risk prediction models (RPMs) have been developed for use in predicting future events in a patients’ journey. RPMs use joint information collected from multiple predictors to provide a prospective insight into future ‘potential’ outcomes. Recent research developments indicate that there is a keen interest amongst researchers to develop RPMs that can be used to predict future events using routinely available information with optimum accuracy. Improvements in the prediction accuracy of RPMs would provide better quality guidance to health care policy makers in decision making process. Most of RPMs suffer from methodological shortcomings due to the inherent heterogeneity which causes patients to have different underlying risk profiles and therefore respond differently to treatment. Ignoring heterogeneity can affect the performance of RPMs which may lead to bias and poor estimation of the underlying risk for individuals. This thesis explores the benefits of using causal reasoning combined with latent variable methods to systematically improve prediction modelling. Throughout the thesis, the potential benefit of incorporating causal assumptions while predicting health outcomes is introduced through a lifecourse perspective using simulated datasets. Specifically, the thesis examines a latent class Cox proportional hazards (PH) model compared to the standard statistical modelling approaches typically adopted that do not explicitly accommodate population heterogeneity. The thesis also compares the Cox neural network approach which uses machine learning principles against the latent class Cox PH model. Lastly, this thesis explores the idea of predicting change, which is a composite outcome, using simulated datasets representing different possible data-generating scenarios and how this can enhance the RPMs.