System Analysis and Design of Physical Delay-Feedback Reservoir Computing Systems

The Reservoir Computing Framework was first developed to aid with training complex recurrent neural networks that exhibit rich dynamical behaviours, and later expanded into a general model for dynamical systems. This allows a complex dynamical system to be treated as a ``black box'' by exploiting its input/output behaviour, which can be trained by a simple training algorithm to apply output weights, making them very computationally efficient. An interesting subset of reservoir computing is the delay-feedback reservoir, which has a hardware efficient architecture by trading space for complexity. A delay-feedback reservoir uses a single non-linear node, a delay-line, and a time-multiplexed input signal to produce a network of ``virtual nodes'', emulating a much larger spatial neural network. While delay-feedback reservoirs have been shown to compute temporal tasks exceptionally well, their design is unclear. Typically, a substrate is found that exhibits interesting dynamical properties, and the delay-feedback reservoir framework is applied, leading to an inefficient use of the substrate. This approach is common, as there is a lack of a general design framework; this is due to the effects of the internal parameters within a delay-feedback reservoir not being well understood. This thesis aims to better understand the function and parameters of a non-linear node within a delay-feedback reservoir so that the reservoir characteristics can be tuned to provide optimal performance for a particular computational task within a particular physical substrate. This thesis examines existing physical implementations of delay-feedback reservoirs to determine the key components within the non-linear node, which is shown to be a non-linear function and an integrator. The effect that the parameters within the key components of the non-linear node have on the performance and system dynamics are extensively investigated, showing that the performance can be significantly improved by changing the timescale of the integrator and by changing the exponent value within the Mackey-Glass non-linear function. An alternative non-linear node is proposed, where the non-linear and integration functions are combined within a high-order filter. An evolutionary algorithm was used to find the optimal filter parameters, which were then evaluated in terms of performance and system metrics. While it was found that a high-order filter can not replace a non-linear node, they can increase the memory of a system and can be used together with a non-linear function to supplement their dynamics.