Multinomial Dirichlet Gaussian Process Model for Classification of Multidimensional Data

Authors: Wanhyun Cho, Soonja Kang, Sangkyoon Kim, Soonyoung Park

Abstract:

We present probabilistic multinomial Dirichlet classification model for multidimensional data and Gaussian process priors. Here, we have considered efficient computational method that can be used to obtain the approximate posteriors for latent variables and parameters needed to define the multiclass Gaussian process classification model. We first investigated the process of inducing a posterior distribution for various parameters and latent function by using the variational Bayesian approximations and important sampling method, and next we derived a predictive distribution of latent function needed to classify new samples. The proposed model is applied to classify the synthetic multivariate dataset in order to verify the performance of our model. Experiment result shows that our model is more accurate than the other approximation methods.

Keywords: Multinomial dirichlet classification model, Gaussian process priors, variational Bayesian approximation, Importance sampling, approximate posterior distribution, Marginal likelihood evidence.

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1110389

References:

[1] C. E. Rasmussen, and C. K. I. Williams, Gaussian Processes for Machine Learning, MIT Press, 2006.
[2] H. Nicklisch, and C. E. Rasmussen, “Approximation for Binary Gaussian process Classification”, Journal of Machine Learning Research, vol 9, pp 2035-75, 2008.
[3] C. K. I. Williams, and D. Barber, “Bayesian Classification with Gaussian Processes”, IEEE Tran. On PAMI, vol 12, pp 1342-1351, 1998.
[4] T. P. Minka, “Expectation Propagation for Approximate Bayesian Inference”, In UAI, Morgan Kaufmann, pp 362-369, 2001.
[5] M. Opper, and O. Winther, “Gaussian Processes for Classification: Mean Field Algorithms”, Neural Computation, vol 12 pp 2655-2204, 2000.
[6] L. Csato, E. Fokoue, M. Opper, and B. Schottky, “Efficient Approaches to Gaussian Process Classification”, In Neural Information Processing Systems, vol 12, pp 251-257, 2000.
[7] M. Girolami and S. Rogers, “Variational Bayesian Multinomial Probit Regression with Gaussian Process Priors”, Neural Computation, vol 18 pp 1790-1817, 2006.