Distributional semantics is fundamental to modern language models and is the basis for learning linguistic vector representations. This leads to errors when encountering phenomena that deviate from standard statistical patterns. This thesis examines two domains presenting semantic outliers: idiomatic expressions and semantic changes in Alzheimer's Disease (AD) speech.
The first part of this work tests two hypotheses around idiomatic expressions: whether high-quality embeddings can be trained using low-resource techniques, and whether modern LLMs have out-of-the-box representations that allow them to surpass smaller fine-tuned models. The latter part explores whether semantic content alone gives sufficient diagnostic signal for AD detection when isolated from surface features.
Idiomatic expressions are non-compositional multiword expressions like 'break a leg', where meaning cannot be derived from component words. We apply existing low resource techniques to idiomaticity tasks. Expression embeddings trained using 1-150 contexts and knowledge injection through Pattern Exploit Training both lead to improvements in English idiomaticity detection, though show weaker results in Portuguese and Galician. Multi-billion parameter LLMs are evaluated on multiple idiomaticity detection datasets, with the best models achieving high F1 scores (>0.86), and analyses revealing strong idiomatic understanding.
AD cognitive impairment manifests as semantic deterioration, including comprehension deficits, semantic paraphasias, and increased generic term usage. We develop an LLM-based pipeline that transforms speech across multiple languages by translating transcripts into English, generating summaries, and creating narrative storyboards. Validation using BLEU, chrF, and semantic similarity shows low surface-form overlap but high semantic preservation. Classifiers trained on transformed transcripts show minimal performance changes (+/-0.1 macro F1), confirming that semantic changes alone provide sufficient diagnostic information.
These findings demonstrate that distributional semantic outliers can both challenge LLMs with idiomaticity, whilst presenting opportunities for AD detection.
