Automated Analysis of Textual Comments in Patient Reported Outcome Measures

Patient-reported outcome measures (PROMs) are questionnaires that capture the patients' perspective on their health-related quality of life. PROMs contain open-ended questions, where they can provide free text comments on information they deem important regarding their outcomes and unmet needs. The PROMs comments are underexplored, largely due to the time and resource demands to analyse them and the current approaches’ inability to scale to large datasets and to compare outcomes across datasets. Addressing these gaps, this thesis aims to automate the analysis of PROMs comments to gain insights into the additional information provided in patients’ free text comments.
The thesis developed a novel patient-centric approach, including quantitative analysis of natural language processing (NLP) models and qualitative feedback from domain experts and patients. Two frameworks have been proposed to gain insights into patient comments from cancer PROMs datasets (prostate and colorectal cancer).
The first original framework is developed to classify free text comments in PROMs with prevalent themes in PROMs identified using a scoping review (Cancer Pathways & Services, Comorbidities, Daily Life, Physical Function, Psychological & Emotional Function, and Social Function). Weakly supervised text classification methods were adopted to label the PROMs comments. The interpretability of the models and the overall utility of the approach have been assessed with PROMs researchers and patients.
The second framework provides a novel application of large language models to summarise groups of PROMs comments. A systematic approach is proposed to generate prompts for summarising groups of PROMs comments and to automatically assess the quality of the resultant summaries. The human evaluation highlights summary features needed to facilitate meaningful analysis.
The approach of this thesis effectively assessed the success of NLP methods to analyse PROMs comments beyond technical robustness, revealing their overall impact and the broader implications for adoption.