A New Weighted LDA Method in Comparison to Some Versions of LDA

Authors: Delaram Jarchi, Reza Boostani

Abstract:

Linear Discrimination Analysis (LDA) is a linear solution for classification of two classes. In this paper, we propose a variant LDA method for multi-class problem which redefines the between class and within class scatter matrices by incorporating a weight function into each of them. The aim is to separate classes as much as possible in a situation that one class is well separated from other classes, incidentally, that class must have a little influence on classification. It has been suggested to alleviate influence of classes that are well separated by adding a weight into between class scatter matrix and within class scatter matrix. To obtain a simple and effective weight function, ordinary LDA between every two classes has been used in order to find Fisher discrimination value and passed it as an input into two weight functions and redefined between class and within class scatter matrices. Experimental results showed that our new LDA method improved classification rate, on glass, iris and wine datasets, in comparison to different versions of LDA.

Keywords: Discriminant vectors, weighted LDA, uncorrelation, principle components, Fisher-face method, Bootstarp method.

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1076314

References:

[1] Xiao-Yuan Jing, David Zhang, and Yuan-Yan Tang ," An improved LDA Approach", IEEE Transaction on Syatems, Man, And CyberneticsÔÇöPart B: Cybernetics, VOL. 34, NO. 5, October 2004.
[2] Yu Bing. Jin Lianfu. Chen Ping,"A new LDA-based method for face recognition", Proceedings of 16th International Conference on Pattern Recognition, Volume 1, 11-15 Aug. 2002 Page(s):168 - 171 vol.. 1.
[3] Tang, E.K. Suganthan, P.N. Yao, X, "Generalized LDA Using Relevance Weighting and evolution strategy", Congress on Evolutionary Computation, 2004. CEC2004. Volume 2, 19-23 June 2004 Page(s): 2230 - 2234 Vol. 2.
[4] P. N. Belhumeur, J. P. Hespanha, and D. J. Kriegman, "Eigenfaces vs. fisherface: Recognition using class specific linear projection," IEEE Trans. Pattern Anal. Machine Intell., vol. 19, pp. 711-720, July 1997.
[5] M. Loog, R.P.W. Duin and R. Haeb-Umbach, ''Multiclass linear dimension reduction by weighted pairwise fisher criteria", IEEE Transaction on Pattern Analysis and Machine Intelligence,23(7),2001,00.762-766.
[6] A. M. Martinez and A. C. Kak, "PCA versus LDA," IEEE Trans. Pattern Anal. Machine Intell., vol. 23, pp. 228-233, Feb. 2001.
[7] Chernick M.R., Bootstrap Methods: A Practitioner-s Guide, John Wiley and Sons, New York, 1999.
[8] Goldberg, D.E. "Genetic algorithms as a computational theory of conceptual design". In Applications of Artificial Intelligence in Engineenng. Vol. 6, 1991, pp, 3-16.
[9] David E. Goldberg, "Genetic Algorithms in Search, Optimization and Machine Learning", Addison-Wesley Longman Publishing Co., Inc., Boston, MA, 1989.