The field of reinforcement learning has seen significant advances in recent years. However, there are still many challenges, including adaptability to environmental changes, robustness to noise, energy efficiency, safety, etc. A promising direction is to incorporate neuroscience findings to explore the potential of replicating the strong cognitive abilities of humans and animals, which, in return, can also contribute to our understanding of brain functions.
In this work, I propose a new model of spiking neural network and derive a reinforcement learning algorithm for it. The algorithm is based on reward-modulated spike-timing-dependent plasticity, thus having better biological plausibility. Experiments on standard reinforcement learning tasks demonstrate its ability to solve challenging tasks and have better inherent robustness to a variety of perturbations than standard methods.
My method is also applied to real-life visual search scanpath modeling tasks that are more challenging. Additionally, I design a new map-based inverse reinforcement learning method that can better extract motivations from scanpaths. Experiments show the effectiveness of the spiking neural network in solving the scanpath modeling task. To obtain an in-depth understanding of the cognitive mechanisms of visual search behaviors, I further apply the reinforcement learning method to the analysis of scanpath properties of social and non-social behaviors of visual search. The results offer new understandings of the patterns of eye movements.
Taken together, the results presented in this thesis provide novel insights into not only developing new reinforcement learning algorithms but also understanding the behaviors and mechanisms of our visual search function.
