Complex-Valued Neural Network in Image Recognition: A Study on the Effectiveness of Radial Basis Function

Authors: Anupama Pande, Vishik Goel

Abstract:

A complex valued neural network is a neural network, which consists of complex valued input and/or weights and/or thresholds and/or activation functions. Complex-valued neural networks have been widening the scope of applications not only in electronics and informatics, but also in social systems. One of the most important applications of the complex valued neural network is in image and vision processing. In Neural networks, radial basis functions are often used for interpolation in multidimensional space. A Radial Basis function is a function, which has built into it a distance criterion with respect to a centre. Radial basis functions have often been applied in the area of neural networks where they may be used as a replacement for the sigmoid hidden layer transfer characteristic in multi-layer perceptron. This paper aims to present exhaustive results of using RBF units in a complex-valued neural network model that uses the back-propagation algorithm (called 'Complex-BP') for learning. Our experiments results demonstrate the effectiveness of a Radial basis function in a complex valued neural network in image recognition over a real valued neural network. We have studied and stated various observations like effect of learning rates, ranges of the initial weights randomly selected, error functions used and number of iterations for the convergence of error on a neural network model with RBF units. Some inherent properties of this complex back propagation algorithm are also studied and discussed.

Keywords: Complex valued neural network, Radial BasisFunction, Image recognition.

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1062540

References:

[1] Amari, S. (1967). ''A theory of adaptive pattern classifier.'' IEEE Transactions on electronic computers, EC-16 (3), 299 - 307.
[2] Benvenunto N. and Piazza F., ''On the Complex Backpropagation Algorithm'', IEEE Trans and Signal Processing, Vol 40, No 4 pp. 967-969, 1992
[3] Derrick, W. R. (1984). ''Complex Analysis and applications.'' Wadsworth, Inc.
[4] Georgiou G.M. and Koutsougeras, ''Complex Domain Backpropagation'', IEEE Trans Circuits and System - II: Analog and Digital Processing, Vol 39, No 5, pp. 330-334, 1992
[5] Hirose. A., ''Proposal of fully Complex-valued Neural Networks'', Proceedings of international Joint Conference on neural networks, Vol 4, pp 152-157, 1992
[6] Kim, M.S., & Guest, C.C. (1990). ''Modification of back propagation networks for complex-valued signal processing in frequency domain.'' Proceedings of the IJNN, San Diego, June, 3, 27-31.
[7] Lipmann R. P., ''An introduction to computing with neural nets'', IEEE Acoustic, Speech and Signal Processing Magzine, pp. 4-22, April 1987.
[8] Mark J. L. Orr (1966, April). ''Introduction to Radial Basis Function Networks.'' Available: http://www.anc.ed.ac.uk/rbf/intro/intro.html
[9] Miyauchi. M.. & Seki. M. (1992a). ''Interpretaion of optical flow through neural network learning.'' Proceedings of IEEE international conference on communication systems, international symposium on information theory and its applications, Singapore. Nov 1247 - 1251
[10] Miyauchi. M., Seki M., Watanabe. A. & Miyauchi. A. (1992 b). ''Interpretation of optical flow through neural network learning''. Proceedings of IAPR workshop on machine vision applications, Tokyo, Dec , 523-528
[11] Nitta T., "An Extension of the Back-Propagation Algorithm to Complex Numbers", Neural Networks, Vol.10, No.8, pp.1391-1415 (1997)
[12] Nitta T., "An Analysis of the Fundamental Structure of Complex-valued Neurons", Neural Processing Letters, Vol.12, No.3, pp.239-246
[13] Nitta T., "Redundancy of the Parameters of the Complex-valued Neural Networks", Neurocomputing, Vol.49, Issue 1-4, pp.423-428 (2002).
[14] Nitta T., "Solving the XOR Problem and the Detection of Symmetry Using a Single Complex-valued Neuron", Neural Networks, Vol.16, No.8, pp.1101-1105 (2003)
[15] P. Geitnner, D. Leers and H.-J. Hagemann: ''Radial Basis Function for speech pattern recognition''
[16] R.A. Finan, A.T. Sapeluk and R.I. Damper, University of Southampton, ''Comparison of Multilayer and Radial Basis Function Networks for Text-Dependent Speaker Recognition''