{"title":"An Improved Learning Algorithm based on the Conjugate Gradient Method for Back Propagation Neural Networks","authors":"N. M. Nawi, M. R. Ransing, R. S. Ransing","country":null,"institution":"","volume":20,"journal":"International Journal of Computer and Information Engineering","pagesStart":2770,"pagesEnd":2775,"ISSN":"1307-6892","URL":"https:\/\/publications.waset.org\/pdf\/446","abstract":"The conjugate gradient optimization algorithm\r\nusually used for nonlinear least squares is presented and is\r\ncombined with the modified back propagation algorithm yielding\r\na new fast training multilayer perceptron (MLP) algorithm\r\n(CGFR\/AG). The approaches presented in the paper consist of\r\nthree steps: (1) Modification on standard back propagation\r\nalgorithm by introducing gain variation term of the activation\r\nfunction, (2) Calculating the gradient descent on error with\r\nrespect to the weights and gains values and (3) the determination\r\nof the new search direction by exploiting the information\r\ncalculated by gradient descent in step (2) as well as the previous\r\nsearch direction. The proposed method improved the training\r\nefficiency of back propagation algorithm by adaptively modifying\r\nthe initial search direction. Performance of the proposed method\r\nis demonstrated by comparing to the conjugate gradient algorithm\r\nfrom neural network toolbox for the chosen benchmark. The\r\nresults show that the number of iterations required by the\r\nproposed method to converge is less than 20% of what is required\r\nby the standard conjugate gradient and neural network toolbox\r\nalgorithm.","references":null,"publisher":"World Academy of Science, Engineering and Technology","index":"Open Science Index 20, 2008"}