Improving the training of Physics Informed Neural Networks via Collocation Point Sampling

Physics-Informed Neural Networks are a subset of machine learning algorithms that are trained by minimising equation residuals, as well as any available data or boundary information. They incorporate physics knowledge by measuring these residuals at collocation points sampled throughout the problem domain. Whilst the cost and accuracy of training is dependent on the number and distribution of these points, optimised sampling procedures have not been widely developed and adopted, with many applications using uniform random distributions or simple residual-driven adaptive sampling.

In this thesis, alternatives to the local residual are implemented as sources of information to guide adaptive resampling of collocation points. The performance of different guiding indicators is evaluated on 1D time‑dependent PDEs, where it is shown that adaptive resampling can reduce the number of collocation points required to reach a given accuracy. Using gradients of the local residual and curvature of solution variables, in particular, is shown to provide greater benefit to accuracy and cost effectiveness.

The methodology is then extended to a system of time‑dependent PDEs in two space dimensions, making necessary adjustments to architecture and training strategy. In this setting, similar improvements to performance are obtained through adaptive resampling, but alternative indicators are found to be less impactful and overall are unable to achieve error levels comparable to the 1D case. Partitioning methods are implemented to improve the robustness of training, but these are not found to circumvent scaling costs, and the use of a candidate set of collocation points, an underlying step in many adaptive sampling procedures and a contributor to this poor scalability, is identified as an important area for improvement in future work.