Efficient Volterra High Order Convolution in Deep Learning and Its Application

Abstract
The capability to capture nonlinear behaviour is essential for achieving strong model
performance. Conventional deep learning architectures typically provide this capability
through the combination of linear convolutional operations and nonlinear components.
In contrast, high order Volterra-based convolution inherently introduces nonlinearity
while simultaneously modelling complex feature interactions (high order terms).
Despite these advantages, its primary drawback is the high computational requirement,
which has hindered its widespread adoption. This challenge is further exacerbated in
deep learning, where multi-dimensional data is commonly encountered. To relieve this
limitation, this thesis proposes constrained and efficient implementations of high order
Volterra-based convolution, which eliminate redundant computations and are adapted
to parallel computational frameworks. Although Volterra series have been researched
for over a century, few studies have investigated their effectiveness in deep learning
model. Therefore, based on proposed implementations, the application of spatial second
order Volterra-based convolution in attention mechanisms is tested in a classification
model. In addition, a variant of this nonlinear convolution is also evaluated on large�scale dataset, with both approaches improving classification performance on small�scale dataset or large-scale dataset. Another constrained nonlinear convolution is also
evaluated in a semantic segmentation model on the binary and multi-class datasets. The
results demonstrate that constrained nonlinear convolution still improves performance
of semantic segmentation.
Main results and findings
• Volterra-based convolution can enhance the performance of existing deep
learning models and attention modules for image classification.
• The proposed implementation performs better than traditional implementations
in terms of performance, training speed, and memory requirements for
classification models.
• Image segmentation models can be enhanced by simply incorporating second
order feature maps.