Probability Density Estimation Using Advanced Support Vector Machines and the Expectation Maximization Algorithm

Authors: Refaat M Mohamed, Ayman El-Baz, Aly A. Farag

Abstract:

This paper presents a new approach for the prob-ability density function estimation using the Support Vector Ma-chines (SVM) and the Expectation Maximization (EM) algorithms.In the proposed approach, an advanced algorithm for the SVM den-sity estimation which incorporates the Mean Field theory in the learning process is used. Instead of using ad-hoc values for the para-meters of the kernel function which is used by the SVM algorithm,the proposed approach uses the EM algorithm for an automatic optimization of the kernel. Experimental evaluation using simulated data set shows encouraging results.

Keywords: Density Estimation, SVM, Learning Algorithms, Parameters Estimation.

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1078625

References:

[1] R. O. Duda, P. E. Hart, and D. G. Stork, Pattern Classification. 2nd ed., Wiley: New York, 2000.
[2] V. Vapnik, The Nature of Statistical Learning Theory. 2nd ed., Springer: New York, 2001.
[3] Refaat M. Mohamed and Aly A. Farag, "Mean Field Theory for Density Estimation Using Support Vector Machines," Seventh International Conference on Information Fusion, Stockholm, July, 2004, pp. 495-501.
[4] V. Vapnik, S. Golowich and A. Smola, "Support Vector Method for Multivariate Density Estimation," Proc. Neural Information Processing Systems 1999 (NIPS 99), Vol. 9, MIT Press:Cambridge, MA, 2000.
[5] B. Scholkopf , C. Burges, and A. Smola, Advances in Kernel Methods:Support Vector Learning. MIT Press:Cambridge, MA, 1999.
[6] Manfred Opper and OleWinther, "Gaussian Processes for Classification: Mean Field Algorithms," Neural Computation., Vol. 12, pp. 2655-2684, 2000.
[7] John W. Lamperti, Probability-A survey of the Mathematical Theory. Wiley Series in probability and Statistics, New York, 1996.