An Exact Solution to Support Vector Mixture

Authors: Monjed Ezzeddinne, Nicolas Lefebvre, Régis Lengellé

Abstract:

This paper presents a new version of the SVM mixture algorithm initially proposed by Kwok for classification and regression problems. For both cases, a slight modification of the mixture model leads to a standard SVM training problem, to the existence of an exact solution and allows the direct use of well known decomposition and working set selection algorithms. Only the regression case is considered in this paper but classification has been addressed in a very similar way. This method has been successfully applied to engine pollutants emission modeling.

Keywords: Identification, Learning systems, Mixture ofExperts, Support Vector Machines.

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1081245

References:

[1] Collobert R. and Bengio S., SVMTorch: "Support Vector Machine for Large-Scale Regression and Classification Problems", Journal of Machine Learning Research, 1: pp. 143-160, 2001.
[2] Genton M., "Classes of Kernels for Machine Learning: A statistics Perspective", Journal of Machine Learning Research 2: pp. 299-312, 2001.
[3] Jacobs R., Jordan M., Nowlan S., and Hinton G., "Adaptative Mixtures of Local Experts", Neural Computation, 3(1): pp. 79 -87, 1991.
[4] Joachims T., Making "Large-Scale SVM Learning Practical", Advances in Kernel Methods - Support Vector Learning, ch. 11, pp. 169-184 , MIT Press, 1999.
[5] Jordan M., Jacobs R., "Hierarchical Mixtures of Experts and the EM Algorithm", Neural Computation, 6(2): pp. 181-214, 1994.
[6] Kwok J., "Support Vector Mixture for Classification and Regression Problems", Proceedings of the International Conference on Pattern Recognition, pp. 255-258, 1998.
[7] Lin C. J., "On the Convergence of the Decomposition Method for Support Vector Machines", IEEE Transactions on Neural Networks, 12 (2001), pp. 1288-1298.
[8] Osuna E., Freund R. and Girosi F., "An Improved Training Algorithm for Support Vector Machines", Proceedings of IEEE Workshop on Neural Networks for Signal Processing pp. 276-285, 1997.
[9] Vapnik V., The Nature of Statistical Learning Theory, Springer, New York, 1995.
[10] Vapnik V., Statistical Learning Theory, A Wiley-Interscience Publication, New York, 1998.