Multi-output Gaussian processes (MOGPs) can concurrently deal with multiple tasks by
exploiting the correlation between different outputs. MOGPs have been mostly used for
multi-output regression datasets, where the responses of each output are continuous values.
However, MOGPs have inferior performance in some complex structured datasets. For
example, MOGPs demand a large computational complexity in large-scale multi-class
classification. The most common type of data in multi-class classification problems consists
of image data, and MOGPs are not specifically designed to handle image datasets so MOGPs
have poor performance on image data that has the nature of high dimensionality. Most
applications of MOGPs are restricted to regression problems with a reduced number of
outputs; and particularly, MOGPs present a limited performance on hierarchical datasets,
i.e., datasets where the observations are connected to each other by means of parent-child
relationships forming a tree structure.
In this thesis, we address the aforementioned issues by proposing three new extensions
of MOGPs separately. First, we develop a novel MOGP model to deal with large-scale multiclass
classification by subsampling both training data sets and classes in each output. Second,
we propose a novel model to deal with image input data sets by incorporating a convolutional
kernel, which can effectively capture information from images, into our developed model
above. Finally, we present a new hierarchical MOGP model with latent variables to handle
hierarchical datasets, where we use a hierarchical kernel function to capture the correlation
within hierarchical data structures and use latent variables to explore dependencies between
outputs.
The new models are applied in various synthetic and real datasets. The results of this thesis
indicate that our proposed models can improve prediction performance in corresponding
datasets.
