A Simple Adaptive Algorithm for Norm-Constrained Optimization

Authors: Hyun-Chool Shin

Abstract:

In this paper we propose a simple adaptive algorithm iteratively solving the unit-norm constrained optimization problem. Instead of conventional parameter norm based normalization, the proposed algorithm incorporates scalar normalization which is computationally much simpler. The analysis of stationary point is presented to show that the proposed algorithm indeed solves the constrained optimization problem. The simulation results illustrate that the proposed algorithm performs as good as conventional ones while being computationally simpler.

Keywords: constrained optimization, unit-norm, LMS, principle component analysis.

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1085700

References:

[1] B. Widrow and S. D. Sterns, Adaptive Signal Processing, Englewood Cliffs, NJ: Prentice Hall, 1985.
[2] A. Benveniste, M. Metivier and P. Priouret, Adapive Algorithms and Stochstic Approximation, New York: Springer-Verlag, 1990.
[3] S. Haykin, Adaptive Filter Theory, Englewood Cliffs, NJ: Prentice Hall, 2002.
[4] A. H. Sayed, Fundamentals of Adaptive Filtering, Englewood Cliffs, NJ: Prentice Hall, 2003.
[5] S. C. Douglas, S. Amari and S. Y. Kung, "On gradient adaptation with uni-norm constraints," IEEE Trans. Signal Processing, vol. 48, no. 6, pp. 1843-1847, June 2000.
[6] S. G. Sankaran and A. A. Louis Beex, "Hyperspherical parametrization for unit-norm based adaptive IIR filtering," IEEE Signal Processing Letters, vol. 6, no. 12, pp. 318-320, Dec. 1999.
[7] R. L'opez-Valcare, "An algorithm for unit-norm equation error system identification based on the method of multipliers," IEEE Tran. on Signal Processing Letters, vol. 6, no. 12, pp. 3080-3085, Dec. 2003.
[8] P. Comon, "Independent component analysis: A new concept?," Signal Processing, vol. 36, no. 3, pp. 287-314, Apr. 1994.
[9] A. Hyvarinen and E. Oja, "Independent component analysis by general nonlinear Hebbian-like learning rules," Signal Processing, vol. 64, no. 3, pp. 301-313, Feb. 1998.