Training deep generative models via lifelong learning

Lifelong learning represents an essential function of an artificial intelligence system, which can continually acquire and learn novel knowledge without forgetting it. Lately, deep learning has brought new possibilities for the development of artificial intelligence, including artificial lifelong learning systems. However, most existing lifelong learning systems are limited to the classification task, and lifelong generative modelling remains a new stage. In this PhD thesis, our research goal mainly focuses on training deep generative models in the context of lifelong learning. The advantage of our research topic over general continual learning is that we can implement many downstream tasks within a unified framework, including classification, image generation, image interpolation, and disentangled representation learning. Firstly, we propose a new lifelong hybrid approach, combining the advantages of Generative Adversarial Net (GAN) and Variational Autoencoder (VAE) for lifelong generative modelling. The proposed model can learn a robust generative replay network that provides high-quality generative replay samples to relieve forgetting, while we also train inference models to capture meaningful latent representations over time. Secondly, to learn a long sequence of tasks, we propose a novel dynamic expansion model that can reuse existing network parameters and knowledge to learn a related task while building a new component to deal with a novel task. Thirdly, we propose a novel lifelong teacher-student framework where a dynamic expansible GAN mixture model implements the teacher module. Then, we introduce a novel self-supervised learning approach for the Student that allows capturing cross-domain latent representations from the entire knowledge accumulated by the Teacher as well as from novel data. Finally, we extend the lifelong teacher-student framework to task-free continual learning, where the task information is unavailable. The proposed model can adaptively expand its network architecture when detecting the data distribution shift during the training, which can be applied to infinite data streams.