Balancing Neural Trees to Improve Classification Performance

Authors: Asha Rani, Christian Micheloni, Gian Luca Foresti

Abstract:

In this paper, a neural tree (NT) classifier having a simple perceptron at each node is considered. A new concept for making a balanced tree is applied in the learning algorithm of the tree. At each node, if the perceptron classification is not accurate and unbalanced, then it is replaced by a new perceptron. This separates the training set in such a way that almost the equal number of patterns fall into each of the classes. Moreover, each perceptron is trained only for the classes which are present at respective node and ignore other classes. Splitting nodes are employed into the neural tree architecture to divide the training set when the current perceptron node repeats the same classification of the parent node. A new error function based on the depth of the tree is introduced to reduce the computational time for the training of a perceptron. Experiments are performed to check the efficiency and encouraging results are obtained in terms of accuracy and computational costs.

Keywords: Neural Tree, Pattern Classification, Perceptron, Splitting Nodes.

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1071178

References:

[1] L. Atlas, R. Cole, Y. Muthusamy, A. Lippman, J. Connor, D. Park, M. El-Sharkawi, and R.J. Marks, A performance comparison of trained multilayer perceptrons and trained classification trees. Proceedings of the IEEE, vol. 78(10), pp. 1614-1619, 1990.
[2] G. Deffuant, Neural units recruitment algorithm for generation of decision trees. Proceedings of the International Joint Conference on Neural Networks, San Diego, CA, vol. 1, pp. 637-642, 1990.
[3] G.L. Foresti and T. Dolso, An Adaptive High-Order Neural Tree for Pattern Recognition, IEEE Trans. on Systemas, Man, Cybernatics- part B: Cybernatics, Vol. 34 (2),pp. 988-996, 2004.
[4] G.L. Foresti and C. Micheloni, Generalized Neural Trees for Pattern Classification, IEEE Trans. on Neural Networks, Vol. 13(6), pp. 1540- 1547, 2002.
[5] G. L. Foresti and G. G. Pieroni, Exploiting neural trees in range image understanding, Pattern Recognit. Lett., vol. 19 (9), pp. 869-878, 1998.
[6] M. Kubat, Decision trees can initialize radial-basis function networks. IEEE Transactions on Neural Networks, vol. 9(5), pp. 813-821, 1998.
[7] T. Li, L. Fang, and A. Jennings,. Structurally adaptive self-organizing neural trees. Proceedings of the International Joint Conference on Neural Networks, Baltimore, MD, vol. 3, pp. 329-334, 1992.
[8] R. Lippmann, An introduction to computing with neural nets, IEEE Acoust. Speech Signal Process. Mag.,vol. 4 (2) pp. 4-22, 1987.
[9] P. Maji, Efficient Design of Neural Network Tree using a Single Spilitting Criterion, Nerocomputing, vol. 71, pp. 787-800, 2008.
[10] P. Maji, C. Shaw, N. Ganguly, B.K. Sikdar, P.P. Chaudhuri, Theory and application of cellular automata for pattern classification, Fundamenta Informaticae, vol. 58(34), pp. 321354, 2003.
[11] D. Michie, D.J. Spiegelhalter and C.C. Taylor, Machine Learning, Neural and Statistical Classification, Ellis Horwood, Chichesten, UK, 1994.
[12] S.K. Murthy,S. Kasif, and S. Salzberg, A system for induction of oblique decision trees. Journal of Artificial Intelligence Research,vol. 2, pp. 1- 32, 1994.
[13] J.R. Quinlan, C4.5: Programs for Machine Learning, Morgan Kaufmann, Los Atlos, CA, 1993.
[14] T.D. Sanger, A tree-structured adaptive network for function approximation in high-dimensional spaces, IEEE Transactions on Neural Networks, vol. 2(2), pp. 285-293, 1991.
[15] A. Sankar, and R.J. Mammone, Optimal pruning of neural tree networks for improved generalization. Proceedings of the International Joint Conference on Neural Networks, Seattle, WA, vol 2, pp 219- 224, 1991.
[16] I.K. Sethi, and J.H. Yoo, Structure-driven induction of decision tree classifiers through neural learning. Pattern Recognition, vol. 30(11), pp. 1893-1904, 1997.
[17] P.E. Utgoff, N.C. Berkman, and J.A. Clouse, Decision tree induction based on efficient tree restructuring. Machine Learning, vol. 29(1), pp. 5-44, 1997.
[18] P.E. Utgoff, and C.E. Brodley, An incremental method for finding multivariate splits for decision trees. Proceedings of the 7th International Conference on Machine Learning, Austin, TX: Morgan Kaufmann, pp. 58-65, 1990.
[19] Z. Zhaou and Z. Chen, Hybrid decision tree, Knowladge based systems, vol. 15(8), pp. 515-528, 2002.