Topic Modelling and Prediction Learning in Small-Sample Text Data: Applications to Geopolitical Forecasting Communities

Geopolitical forecasting platforms such as Metaculus and the Good Judgment Project rely on collective intelligence to produce probabilistic predictions about uncertain global events. In addition to numerical forecasts, participants generate extensive textual reasoning describing the evidence and causal interpretations underlying their judgements. These texts potentially contain valuable information about forecasting behaviour, yet systematically extracting quantitative signals from them remains challenging. Forecasting comments typically form high-dimensional sparse count representations observed under small-sample regimes, which leads to unstable estimation of latent semantic structures.
This thesis develops a unified statistical pipeline for analysing textual information in collective forecasting environments.
Built on low-rank matrix factorisation, it enables stable estimation of semantic representations under sparse conditions while supporting temporal modelling and predictive integration. First, a comparative study of classical topic modelling approaches evaluates their ability to recover latent semantic structures from sparse short-text corpora. Second, a temporally regularised dynamic nonnegative matrix factorisation (NMF) model is introduced to capture semantic evolution over time. The regularisation penalises abrupt changes between adjacent topic representations, enforcing temporal continuity and dampening stochastic estimation fluctuations. Third, a supervised NMF framework is introduced to jointly estimate latent semantic structures and numerical forecasting outcomes, establishing a direct statistical mapping between textual semantic
representations and numerical forecasting outcomes.
Empirical analyses using data from geopolitical forecasting communities show that the proposed approach stabilises latent semantic structure estimation, identifies structural shifts and improves out-of-sample predictive performance. The results demonstrate that textual reasoning constitutes a valuable statistical resource for understanding and improving collective forecasting systems.