Multi-output Gaussian Processes for Large-scale Multi-class Classification and Hierarchical Data

Multi-output Gaussian processes (MOGPs) can concurrently deal with multiple tasks by exploiting the correlation between different outputs. MOGPs have been mostly used for multi-output regression datasets, where the responses of each output are continuous values. However, MOGPs have inferior performance in some complex structured datasets. For example, MOGPs demand a large computational complexity in large-scale multi-class classification. The most common type of data in multi-class classification problems consists of image data, and MOGPs are not specifically designed to handle image datasets so MOGPs have poor performance on image data that has the nature of high dimensionality. Most applications of MOGPs are restricted to regression problems with a reduced number of outputs; and particularly, MOGPs present a limited performance on hierarchical datasets, i.e., datasets where the observations are connected to each other by means of parent-child relationships forming a tree structure. In this thesis, we address the aforementioned issues by proposing three new extensions of MOGPs separately. First, we develop a novel MOGP model to deal with large-scale multiclass classification by subsampling both training data sets and classes in each output. Second, we propose a novel model to deal with image input data sets by incorporating a convolutional kernel, which can effectively capture information from images, into our developed model above. Finally, we present a new hierarchical MOGP model with latent variables to handle hierarchical datasets, where we use a hierarchical kernel function to capture the correlation within hierarchical data structures and use latent variables to explore dependencies between outputs. The new models are applied in various synthetic and real datasets. The results of this thesis indicate that our proposed models can improve prediction performance in corresponding datasets.