Learning the Evolution of Time-dependent Partial Differential Equations

This thesis investigates the challenges of using deep learning models, specifically Neural Operators, to predict the long-term temporal evolution of complex physical systems governed by Partial Differential Equations (PDEs). While deep learning surrogate models can offer significant computational speedups over traditional numerical solvers, they frequently suffer from autoregressive error accumulation (exposure bias) and a lack of temporal flexibility when constrained to fixed timesteps during inference. This research addresses these limitations through three primary contributions.

First, we develop Scheduled Temporal Loss Weighting (STLW), a novel curriculum learning strategy for constant-timestep models. STLW dynamically adjusts loss weights across a temporal rollout during training, prioritising short-term accuracy initially and
gradually increasing the influence of long-term gradients. Experimental results demonstrate that STLW significantly reduces error propagation, achieving up to a 62.5% reduction in error for the viscous Burgers’ equation compared to standard baselines.

Second, we introduce an attention-based temporal conditioning mechanism to break the dependence on fixed time-stepping. By integrating cross-attention with temporal embeddings, a single Fourier Neural Operator (FNO) can predict at variable timesteps.

Third, this work demonstrates the feasibility of variable-timestep sampling strategies, establishing that “large-to-small” or “small to-large” stepping can effectively bypass high-error phases in PDE evolution depending on the physical regime.

The proposed methods are rigorously validated across a diverse suite of benchmark PDEs, including Advection, Viscous and Inviscid Burgers’, Diffusion-Reaction, Diffusion-Sorption, and Korteweg-de Vries(KdV) equations. Overall, the results demonstrate that combining architectural flexibility with advanced training strategies can yield superior global stability and generalisation in long-term prediction.