Kernel function, which allows the formulation of nonlinear variants of any algorithm that can be cast in terms of dot products, makes the Support Vector Machines (SVM) have been successfully applied in many fields, e.g. classification and regression. The importance of kernel has motivated many studies on its composition. It-s well-known that reproducing kernel (R.K) is a useful kernel function which possesses many properties, e.g. positive definiteness, reproducing property and composing complex R.K by simple operation. There are two popular ways to compute the R.K with explicit form. One is to construct and solve a specific differential equation with boundary value whose handicap is incapable of obtaining a unified form of R.K. The other is using a piecewise integral of the Green function associated with a differential operator L. The latter benefits the computation of a R.K with a unified explicit form and theoretical analysis, whereas there are relatively later studies and fewer practical computations. In this paper, a new algorithm for computing a R.K is presented. It can obtain the unified explicit form of R.K in general reproducing kernel Hilbert space. It avoids constructing and solving the complex differential equations manually and benefits an automatic, flexible and rigorous computation for more general RKHS. In order to validate that the R.K computed by the algorithm can be used in SVM well, some illustrative examples and a comparison between R.K and Gaussian kernel (RBF) in support vector regression are presented. The result shows that the performance of R.K is close or slightly superior to that of RBF.<\/p>\r\n","references":"[1] J. Mercer, Ed., Functions of Positive and Negative Type and Their\r\nConnection with The Theory of Integral Equations, ser. Philosophical\r\nTransactions of the Royal Society, London, 1909, vol. A, 209.\r\n[2] V. Vapnik, The Nature of Statistical Learning Theory. New York:\r\nSpringer-Verlag, 1995.\r\n[3] G. Bloch, F. Lauer, G. Colin, and Y. Chamaillard, \"Support vector regression\r\nfrom simulation data and few experimental samples,\" Information\r\nSciences, vol. 178, pp. 3813-3827, 2008.\r\n[4] V. Blanz, B. Sch\u252c\u00bfolkopf, H. B\u252c\u00bfulthoff, and C. J. Burges, \"Comparison of\r\nview-based object recognition algorithms using realistic 3d models,\" in\r\nArtificial Neural Networks\u253c\u00e9ICANN-96, C. v. d. Malsburg, W. v. Seelen,\r\nJ. C. Vorbr\u252c\u00bfuggen, and B. Sendhoff, Eds., vol. 1112. Berlin: Springer\r\nLecture Notes in Computer Science, 1996.\r\n[5] C. J. Burges and B. Sch\u252c\u00bfolkopf, \"Improving the accuracy and speed of\r\nsupport vector learning machines,\" in Advances in Neural Information\r\nProcessing Systems, vol. 9. Cambridge, MA: MIT Press, 1997, pp.\r\n375-381.\r\n[6] B. Sch\u252c\u00bfolkopf, K.-K. Sung, C. J. Burges, F. Girosi, P. Niyogi, T. Poggio,\r\nand V. Vapnik, \"Comparing support vector machines with gaussian\r\nkernels to radial basis function classifiers,\" IEEE Transactions on Signal\r\nProcessing, vol. 45, pp. 2758-2765, 1997.\r\n[7] E. E. Osuna, R. Freund, and F. Girosi, \"Training support vector\r\nmachines: An application to face detection,\" in IEEE Conference on\r\nComputer Vision and Pattern Recognition, 1997, pp. 130-136.\r\n[8] W. Kienzle, G. Bakir, M. Franz, and B. Sch\u252c\u00bfolkopf, \"Face detection\u253c\u00e9efficient\r\nand rank deficient,\" in Advances in Neural Information Processing\r\nSystems, Y. Weiss, Ed., vol. 17. MIT Press, 2005, pp. 673-680.\r\n[9] J.-B. Gao, S. R. Gunn, and C. J. Harris, \"Mean field method for the\r\nsupport vector machine regression,\" Neurocomputing, vol. 50, pp. 391-\r\n405, 2003.\r\n[10] M. A. Mohandes, T. O. Halawani, S. Rehman, and A. A. Hussain, \"Support\r\nvector machines for wind speed prediction,\" Renewable Energy,\r\nvol. 29, no. 6, pp. 939-947, 2004.\r\n[11] W.-W. He, Z.-Z. Wang, and H. Jiang, \"Model optimizing and feature\r\nselecting for support vector regression in time series forecasting,\"\r\nNeurocomputing, vol. 73, no. 3, pp. 600-611, 2008.\r\n[12] F. Pan, P. Zhu, and Y. Zhang, \"Metamodel-based lightweight design of\r\nb-pillar with twb structure via support vector regression,\" Computers\r\nand Structures, vol. 88, pp. 36-44, 2010.\r\n[13] C. J. Burges, \"A tutorial on support vector machines for pattern recognition,\"\r\nData Mining and Knowledge Discovery, vol. 2, pp. 121-167,\r\n1998.\r\n[14] A. J. Smola and B. Sch\u252c\u00bfolkopf, \"A tutorial on support vector regression,\"\r\nStatistics and Computing, vol. 14, no. 3, pp. 199-222, 2004.\r\n[15] B. E. Boser, I. M. Guyon, and V. Vapnik, \"A training algorithm for\r\noptimal margin classifiers,\" in Proceedings of the 5th Annual ACM\r\nWorkshop on Computational Learning Theory, D. Haussler, Ed. ACM\r\nPress, 1992, pp. 144-152.\r\n[16] B. Sch\u252c\u00bfolkopf, \"The kernel trick for distances,\" Neural Information\r\nProcess. Systems (NIPS), vol. 13, 2000.\r\n[17] D. Anguita and G. Bozza, \"The effect of quantization on support vector\r\nmachines with gaussian kernel,\" in Proceedings of International Joint\r\nConference on Neural Networks, Montreal, Canada, 2005, pp. 681-684.\r\n[18] X.-Y. Zhang and Y.-C. Liu, \"Performance analysis of support vector\r\nmachines with gauss kernel,\" Computer Engineering, vol. 29, no. 8, pp.\r\n22-25, 2003.\r\n[19] Y. Tan and J. Wang, \"A support vector machine with a hybrid kernel\r\nand minimal vapnik-chervonenkis dimension,\" IEEE Transactions on\r\nKnowledge and Data Engineering, vol. 16, pp. 385-395, 2004.\r\n[20] J.-X. Liu, J. Li, and Y.-J. Tan, \"An empirical assessment on the\r\nrobustness of support vector regression with different kernels,\" in\r\nProceedings of the 4th International Conference on Machine Learning\r\nand Cybernetics, vol. 7, Guangzhou, China, 2005.\r\n[21] R. Opfer, \"Multiscale kernels,\" Advances in Computational Mathematics,\r\nvol. 25, pp. 357-380, 2006.\r\n[22] L. Zhang, W.-D. Zhou, and L.-C. Jiao, \"Wavelet support vector machine,\"\r\nIEEE Transactions on Systems, Man and Cybernetics - Part B:\r\nCybernetics, vol. 34, pp. 34-39, 2004.\r\n[23] X.-G. Zhang, D. Gao, X.-G. Zhang, and S.-J. Ren, \"Robust wavelat\r\nsupport machines for regression estimation,\" International Journal Information\r\nTechnology, vol. 11, no. 9, pp. 35-46, 2005.\r\n[24] A. J. Smola, B. Sch\u252c\u00bfolkopf, and K.-R. M\u252c\u00bfuller, \"The connection between\r\nregularization operators and support vector kernels,\" Neural Networks,\r\nvol. 11, pp. 637-649, 1998.\r\n[25] R. Schaback and H. Wendland, \"Approximation by positive definite\r\nkernels,\" in Advanced Problems in Constructive Approximation, M. D.\r\nBuhmann and D. H. Mache, Eds. Birkh\u252c\u00bfauser, Basel: Verlag, 2002, pp.\r\n203-221.\r\n[26] G. Wahba, \"Support vector machines, reproducing kernel hilbert spaces\r\nand randomized gacv,\" in Advances in Kernel Methods-Support Vector\r\nLearning, B. Sch\u252c\u00bfolkopf, C. J. Burges, and A. J. Smola, Eds. Cambridge,\r\nEngland: MIT Press, 1999, pp. 69-88.\r\n[27] L.-M. Ma and Z.-M. Wu, \"Kernel based approximation in sobolev spaces\r\nwith radial basis functions,\" Applied Mathematics and Computation, vol.\r\n215, p. 2229C2237, 2009.\r\n[28] N. Aronszajn, \"Theory of reproducing kernels,\" Transactions of the\r\nAmerican Mathematical Society, vol. 68, no. 3, pp. 337-404, 1950.\r\n[29] J. Gao, C. J. Harris, and S. R. Gunn, \"Support vector kernel based on\r\nframes in function hilbert spaces,\" Neural Computation, vol. 13, pp.\r\n1975-1994, 2001.\r\n[30] R. Schaback, \"A unified theory of radial basis functions native hilbert\r\nspaces for radial basis functions ii,\" Journal of Computational and\r\nApplied Mathematics, vol. 121, pp. 165-177, 2000.\r\n[31] S. Bergman, \"The approximation of functions satisfying a linear partial\r\ndifferential equation,\" Duke Mathematics Journal, vol. 6, pp. 537-561,\r\n1940.\r\n[32] M.-G. Cui and Z.-X. Deng, \"On the best operator of interpolation,\"\r\nMath. Numerica Sinica, vol. 8, no. 2, pp. 209-216, 1986.\r\n[33] J. Ling and Y.-S. Li, \"A new method for computing reproducing kernels,\"\r\nNortheast Math. J., vol. 14, no. 4, pp. 467-473, 1998.\r\n[34] B. Sch\u252c\u00bfolkopf, A. J. Smola, and K.-R. M\u252c\u00bfuller, \"Nonlinear component\r\nanalysis as a kernel eigenvalue problem,\" Neural Computation, vol. 10,\r\nno. 5, pp. 1299-1319, 1998.\r\n[35] S. Mika, G. Ratsch, B. Weston, B. Sch\u252c\u00bfolkopf, and K.-R. M\u252c\u00bfuller, \"Fisher\r\ndiscriminate analysis with kernels,\" in Neural Networks for Signal\r\nProcessing IX, Y. H. Hu, J. Larsen, E. Wilson, and S. Douglas, Eds.\r\nIEEE Press, 1999, pp. 41-48.\r\n[36] D. MacDonald and C. Fyfe, \"The kernel self-organizing map,\" in\r\nProceedings of 4th International Conference on Knowledge-based Intelligent\r\nEngineering Systems and Allied Technologies, KES 2000, R. J.\r\nHowlett and L. C. Jain, Eds., vol. 1, 2000, pp. 317-320.\r\n[37] A. J. Smola, B. Sch\u252c\u00bfolkopf, and K.-R. M\u252c\u00bfuller, \"General cost functions\r\nfor support vector regression,\" in Proceedings of Ninth Australian Conf.\r\non Neural Networks, 1998, pp. 79-83.\r\n[38] S. Bochner, Lectures on Fourier Integral. Princeton, New Jersey:\r\nPrinceton University Press, 1959.\r\n[39] A. J. Smola, Z. L. O' va'ri, and R. C. Williamson, \"Regularization with\r\ndot-product kernels,\" in Advances in Neural Information Processing\r\nSystems, T. K. Leen, T. G. Dietterich, and V. Tresp, Eds., vol. 13. MIT\r\nPress, 2001, pp. 308-314.\r\n[40] A. Berlinet and C. Thomas-Agnan, Reproducing Kernel Hilbert Spaces\r\nin Probability and Statistics. Boston, Dordrecht, London: Kluwer\r\nAcademic Publishers Group, 2003.\r\n[41] N. Cristianini and J. Shawe-Taylor, An Introduction to Support Vector\r\nMachines and Other Kernel-based Learning Methods. Cambridge, U.K:\r\nCambridge University Press, 2000.\r\n[42] M.-G. Cui and Y.-Z. Lin, Nonlinear Numerical Analysis in Reproducing\r\nKernel Space. Commack, NY: Nova Science Publishers, Inc., 2009.\r\n[43] M.-G. Cui, M. Zhang, and Z.-X. Deng, \"Two-dimensional reproducing\r\nkernel and surface interpolation,\" Journal Computational Mathematic,\r\nvol. 4, no. 2, pp. 177-181, 1986.\r\n[44] W. Zhang, \"The construction of reproducing kernel and some approximating\r\nproblems in the reproducing kernel spaces,\" Ph.D. dissertation,\r\nNational University of Defense Technology, Changsha, Hunan, China,\r\n2005.\r\n[45] X.-J. Zhang and H. Long, \"Computing reproducing kernels for wm\r\n2 (a, b) (i),\" Mathematic Numerica Sinica, vol. 30, no. 3, pp. 295-304, 2008.\r\n[46] Y.-S. Li, \"On the recurrence relations for b-splines defined by certain\r\nl-splines,\" Journal of Approximation theory, vol. 43, pp. 359-369, 1985.\r\n[47] V. E. Neagoe, \"Inversion of the van der monde matrix,\" IEEE Signal\r\nProcessing Letters, vol. 3, no. 4, pp. 119-120, 1996.","publisher":"World Academy of Science, Engineering and Technology","index":"Open Science Index 39, 2010"}