Graphical Summarisation of Argumentative Text

This work investigates the summarisation of argumentative text. Our main focus is on the generation of graphical summaries from dialogical argumentative texts such as online news comment sections.

We develop a novel type of graph structure to summarise argument, which we refer to as Argument Summary Graphs or ASGs. Our contributions are twofold. Firstly, we develop two new data resources to investigate the generation of ASGs, the Debatabase-ASG dataset (created from a curated collection of online debates), and the SENSEI-ASG dataset, developed by adding annotations to the SENSEI dataset of online news article comments.

Secondly, we investigate alternative methods for generating ASGs from both datasets using Large Language Models (LLMs). We carry out experiments on the Debatabase-ASG dataset and find that an end-to-end text-to-text method performs better than a pipeline approach. Additionally, we test how well the end-to-end approach generalises across corpora, using both of the corpora we created, and a third argument mining corpus, the Argument Annotated Essays corpus (AAEC). We find that additional fine-tuning on a monological dataset from a distinct Argument Mining task provides similar benefits to fine-tuning on a second in-genre dataset.

We also carry out two strands of work not narrowly focused on ASGs. Firstly, we investigate the prediction of Reasoning Markers, used for detection of argumentative text. We create a corpus for this task and implement multiple baselines, with the best achieving an F1 score of 0.69. Secondly, we investigate Argument Structure Parsing: the task of extracting argumentative components and their relations from text. We fine-tune several LLMs which achieve near state-of-the-art scores in the end-to-end setting. We propose a novel method of formatting the output for this task, which we find competitive with the existing state-of-the-art approach in evaluation metrics, while being significantly faster to generate.