Reservoir Computing (RC) is an unconventional computing model first designed for training Recurrent Neural Networks (RNNs). The technique involves randomly initiating the weights of the inner neural network (called a reservoir), then training the observed output of this reservoir using a single-layer readout.
The design of the reservoir computing model allows us to treat the reservoir as a black box. This makes RC ideal for computing with unconventional physical materials, as we can treat the material as a black box reservoir. This physical or ``in materio'' reservoir computing allows us to exploit the dynamics of physical systems, and make strides towards smaller-scale, low power computing.
One challenge in the field of in materio RC is that the materials with the most interesting dynamics are difficult or impossible to work with at large scale. In this thesis, we study how we might scale up reservoir computers by combining multiple smaller reservoirs together.
Our approach of combining reservoirs allows us to have reservoirs with different properties, such as distinct materials or timescales. This approach allows us to tackle more complex tasks than are typically possible with classical RC, such as the sleep apnea benchmark.
Our work is completed in simulation. In service of this, we work to bring our simulation closer to the constraints of physical RC with ``mock materials'' we design. We design a technique for building heterogeneous reservoirs for complex tasks.
We find that heterogeneous reservoirs are not suited to all RC tasks, such as a variation we design of the Multiple Superimposed Oscillators (MSO*) benchmark. We propose alternative reservoir designs which may be found to be more effective in future work.
Data and code related to this thesis is available from https://github.com/FromAnkyra.
ORCID: https://orcid.org/0000-0002-5764-2181
(2024)
Scaling up ESNs with Heterogeneous Reservoirs.
PhD thesis, University of York.
