An Iterative Algorithm for KLDA Classifier

Authors: D.N. Zheng, J.X. Wang, Y.N. Zhao, Z.H. Yang

Abstract:

The Linear discriminant analysis (LDA) can be generalized into a nonlinear form - kernel LDA (KLDA) expediently by using the kernel functions. But KLDA is often referred to a general eigenvalue problem in singular case. To avoid this complication, this paper proposes an iterative algorithm for the two-class KLDA. The proposed KLDA is used as a nonlinear discriminant classifier, and the experiments show that it has a comparable performance with SVM.

Keywords: Linear discriminant analysis (LDA), kernel LDA (KLDA), conjugate gradient algorithm, nonlinear discriminant classifier.

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1059321

References:

[1] B. Scholköpf, A. Smola, and K.R. M├╝ller, "Nonlinear Component Analysis as a Kernel Eigenvalue Problem," Neural Computation, vol.10, pp. 1299-1319, 1998.
[2] G. Baudat and F. Anouar, "Generalized Discriminant Analysis Using a Kernel Approach," Neural Computation, vol.12, pp. 2385-2404, 2000.
[3] C. Park and H. Park, "Fingerprint Classification Using Nonlinear Discriminant Analysis," Technical Report, TR 03-034, University of Minnesota, USA, Sep. 2003.
[4] S. Mika, G. R├ñtsch, J. Weston, B. Schölkopf, and K.R. M├╝ller, "Fisher discriminant analysis with kernels," Neural Networks for Signal Processing IX, pp. 41-48, 1999.
[5] P.N. Belhumeour, J.P. Hespanha, and D.J. Kriegman, "Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection," IEEE Trans. Pattern Analysis and Machine Intell., vol.19, pp. 711-720, 1997.
[6] H.C. Kim, D. Kim, and S.Y. Bang, "Face recognition using LDA mixture model," Pattern Recognition Letters, vol.24, pp. 2815-2821, 2003.
[7] G. Rätsch, T. Onoda, and K.R. Müller, "Soft Margins for AdaBoost," Machine Learning, pp. 1-35, 2000.
[8] C. Burges, "Simplified Support Vector Decision Rules," in Proceedings of the 13th International Conference on Machine Learning, pp. 71-77, 1996.
[9] B. Schölkopf, P. Knirsch, A. Smola, and C. Burges, "Fast Approximation of Support Vector Kernel Expansions, and an Interpretation of Clustering as Approximation in Feature Spaces," Proceedings of the DAGM Symposium Mustererkennung, pp. 124-132, 1998.