Can Attention-Inspired Artificial Intelligence Provide a Diagnostic Understanding of Colorectal Cancer Imaging Data?

Digital pathology workflows provide high-resolution whole slide images (WSIs) for assessing diseases such as cancer at a cellular level. A worldwide shortage of pathologists limits the adoption of labour-intensive analysis techniques. This is potentially a role for Artificial Intelligence (AI). However, current AI models need images in the order of 200x200 pixels, while WSIs are of gigapixel size. AI-based systems must address this scale discrepancy without overlooking diagnostically important features in the WSI.
Work in this thesis was motivated by human visual attention, where relevant features of an input scene are selected in response to goals in executive brain regions, avoiding processing the whole scene at full resolution.
Two novel WSI processing pipelines incorporated attention-like algorithms. The first used a thumbnail image to map tumour density, controlling the sampling density of full-magnification patches for classification with a convolutional neural network (CNN). A later pipeline introduced weighted regular sampling (WRS) to mitigate sampling biases. The estimated class distributions yielded the tumour outline and tumour stroma ratio (TSR), a predictor of disease severity.
A novel Feedback Attention Ladder CNN (FAL-CNN) used feedback attention, significantly increasing classification accuracy from 79.33% to 82.82% (p<0.001) with 9-class colorectal cancer patches. Top-to-bottom and local-group feedback were combined to generate attention masks for the forward path. Increased accuracy with ImageNet-100 showed the approach to be transferrable. In the WRS pipeline, TSR error was substantially reduced at pathologist-selected locations, suggesting application in a TSR measurement tool.
Visualisations of attention masks in the FAL-CNN highlighted informative tissue regions. A novel saccade model resampled the input patch to align the centre-focused FAL-CNN on these regions. The model discovered salient features even when outside the initial patch. Pathologist relabelling of resampled patches confirmed the saccade model’s ability to locate nearby regions of tumour, a potentially valuable behaviour in cancer WSI analysis.