Minimal Residual Method for Adaptive Filtering with Finite Termination

Authors: Noor Atinah Ahmad, Shazia Javed

Abstract:

We present a discussion of three adaptive filtering algorithms well known for their one-step termination property, in terms of their relationship with the minimal residual method. These algorithms are the normalized least mean square (NLMS), Affine Projection algorithm (APA) and the recursive least squares algorithm (RLS). The NLMS is shown to be a result of the orthogonality condition imposed on the instantaneous approximation of the Wiener equation, while APA and RLS algorithm result from orthogonality condition in multi-dimensional minimal residual formulation. Further analysis of the minimal residual formulation for the RLS leads to a triangular system which also possesses the one-step termination property (in exact arithmetic)

Keywords: Adaptive filtering, minimal residual method, projection method.

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1071552

References:

[1] B. Farhang-Boroujeny, Adaptive filters: theory and applications, John Wiley & Sons, Inc.,1998.
[2] G. K. Boray and M. D. Srinath, Conjugate gradient techniques for adaptive filtering, IEEE Trans. Circuits Syst., 1992.
[3] P. S. Chang and A. N. Wilson, Jr., Analysis of conjugate gradient method for adaptive filtering, IEEE Trans. Signal Processing, 2000.
[4] M. Q. Chen, A direction set based algorithm for least squares problems in adaptive signal processing, Linear Algebra and its Application, 1998.
[5] G. F. Xu, T. Bose and J. Schroeder, The Euclidean direction search algorithm for adaptive filtering, Proceedings of the 1999 IEEE Int. Symp. on Circuits and Systems, 1999.
[6] N. A. Ahmad, Accelerated euclidean direction search algorithm and related relaxation schemes for solving adaptive filtering problem, Proceedings of IEEE Int. Conf. on Information, Communication and Signal Processing (ICICS 2007), 2007.
[7] N. A. Ahmad, A globally convergent stochastic pairwise conjugate gradient based adaptive filtering algorithm, IEEE Signal Process. Letters, 2008.
[8] P. S. R. Diniz, M. L. R. de Campos and A. Antoniou, Analysis of LMSNewton Adaptive Filtering Algorithms with Variable Convergence Factor, IEEE Trans. Signal Processing, 1995.
[9] C. W. Lee, Y. K. Lee and S. W. Kim, An Affine Projection Algorithm with a Data-Selective Method of using the Condition Number, Signal Processing 88, 2008.
[10] Y. Saad, Iterative Methods for Sparse Linear Systems, Englewood Cliffs, NJ: Prentice-Hall, 1991.
[11] S. Haykin, Adaptive Filter Theory, 2nd edition, Prentice Hall,1991.
[12] P. S. R. Diniz,Adaptive Filtering: Algorithms and Practical Implementation, 3rd Edition, Springer, 2008.