Search results for: Fast Neural Networks

Authors: Seyed Fazel Ziaei Asl, Ali Karami, Gholamreza Ashari, Azam Behrang, Arezoo Assareh, N.Hedayat

Abstract:

Predict daily global solar radiation (GSR) based on meteorological variables, using Multi-layer perceptron (MLP) neural networks is the main objective of this study. Daily mean air temperature, relative humidity, sunshine hours, evaporation, wind speed, and soil temperature values between 2002 and 2006 for Dezful city in Iran (32° 16' N, 48° 25' E), are used in this study. The measured data between 2002 and 2005 are used to train the neural networks while the data for 214 days from 2006 are used as testing data.

Keywords: Multi-layer Perceptron (MLP) Neural Networks;Global Solar Radiation (GSR), Meteorological Parameters, Prediction.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 2928

Authors: Marina Velikova, Hennie Daniels, Ad Feelders

Abstract:

In many applications, it is a priori known that the target function should satisfy certain constraints imposed by, for example, economic theory or a human-decision maker. Here we consider partially monotone problems, where the target variable depends monotonically on some of the predictor variables but not all. We propose an approach to build partially monotone models based on the convolution of monotone neural networks and kernel functions. The results from simulations and a real case study on house pricing show that our approach has significantly better performance than partially monotone linear models. Furthermore, the incorporation of partial monotonicity constraints not only leads to models that are in accordance with the decision maker's expertise, but also reduces considerably the model variance in comparison to standard neural networks with weight decay.

Keywords: Mixture models, monotone neural networks, partially monotone models, partially monotone problems.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 1567

Authors: Xinhua Zhang, Kelin Li

Abstract:

In this paper, the problem of stability analysis for a class of impulsive stochastic fuzzy neural networks with timevarying delays and reaction-diffusion is considered. By utilizing suitable Lyapunov-Krasovskii funcational, the inequality technique and stochastic analysis technique, some sufficient conditions ensuring global exponential stability of equilibrium point for impulsive stochastic fuzzy cellular neural networks with time-varying delays and diffusion are obtained. In particular, the estimate of the exponential convergence rate is also provided, which depends on system parameters, diffusion effect and impulsive disturbed intention. It is believed that these results are significant and useful for the design and applications of fuzzy neural networks. An example is given to show the effectiveness of the obtained results.

Keywords: Exponential stability, stochastic fuzzy cellular neural networks, time-varying delays, impulses, reaction-diffusion terms.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 1332

Authors: Zixin Liu, Huawei Yang, Fangwei Chen

Abstract:

This paper studies the mean square exponential synchronization problem of a class of stochastic neutral type chaotic neural networks with mixed delay. On the Basis of Lyapunov stability theory, some sufficient conditions ensuring the mean square exponential synchronization of two identical chaotic neural networks are obtained by using stochastic analysis and inequality technique. These conditions are expressed in the form of linear matrix inequalities (LMIs), whose feasibility can be easily checked by using Matlab LMI Toolbox. The feedback controller used in this paper is more general than those used in previous literatures. One simulation example is presented to demonstrate the effectiveness of the derived results.

Keywords: Exponential synchronization, stochastic analysis, chaotic neural networks, neutral type system.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 1515

Authors: Benedek Kovacs, Janos Toth

Abstract:

Solutions are proposed for the central problem of estimating the reaction rate coefficients in homogeneous kinetics. The first is based upon the fact that the right hand side of a kinetic differential equation is linear in the rate constants, whereas the second one uses the technique of neural networks. This second one is discussed deeply and its advantages, disadvantages and conditions of applicability are analyzed in the mirror of the first one. Numerical analysis carried out on practical models using simulated data, and our programs written in Mathematica.

Keywords: Neural networks, parameter estimation, linear regression, kinetic models, reaction rate coefficients.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 1932

Authors: Vinay Chandwani, Vinay Agrawal, Ravindra Nagar

Abstract:

Artificial Neural Networks (ANN) trained using backpropagation (BP) algorithm are commonly used for modeling material behavior associated with non-linear, complex or unknown interactions among the material constituents. Despite multidisciplinary applications of back-propagation neural networks (BPNN), the BP algorithm possesses the inherent drawback of getting trapped in local minima and slowly converging to a global optimum. The paper present a hybrid artificial neural networks and genetic algorithm approach for modeling slump of ready mix concrete based on its design mix constituents. Genetic algorithms (GA) global search is employed for evolving the initial weights and biases for training of neural networks, which are further fine tuned using the BP algorithm. The study showed that, hybrid ANN-GA model provided consistent predictions in comparison to commonly used BPNN model. In comparison to BPNN model, the hybrid ANNGA model was able to reach the desired performance goal quickly. Apart from the modeling slump of ready mix concrete, the synaptic weights of neural networks were harnessed for analyzing the relative importance of concrete design mix constituents on the slump value. The sand and water constituents of the concrete design mix were found to exhibit maximum importance on the concrete slump value.

Keywords: Artificial neural networks, Genetic algorithms, Back-propagation algorithm, Ready Mix Concrete, Slump value.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 2847

Authors: K. Sreelakshmi, P. Ramakanthkumar

Abstract:

Predicting short term wind speed is essential in order to prevent systems in-action from the effects of strong winds. It also helps in using wind energy as an alternative source of energy, mainly for Electrical power generation. Wind speed prediction has applications in Military and civilian fields for air traffic control, rocket launch, ship navigation etc. The wind speed in near future depends on the values of other meteorological variables, such as atmospheric pressure, moisture content, humidity, rainfall etc. The values of these parameters are obtained from a nearest weather station and are used to train various forms of neural networks. The trained model of neural networks is validated using a similar set of data. The model is then used to predict the wind speed, using the same meteorological information. This paper reports an Artificial Neural Network model for short term wind speed prediction, which uses back propagation algorithm.

Keywords: Short term wind speed prediction, Neural networks, Back propagation.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 3018

Authors: S. Mousavian, D. Ashouri, F. Mousavian, V. Nikkhah Rashidabad, N. Ghazinia

Abstract:

PH, temperature and time of extraction of each stage,  agitation speed and delay time between stages effect on efficiency of  zinc extraction from concentrate. In this research, efficiency of zinc  extraction was predicted as a function of mentioned variable by  artificial neural networks (ANN). ANN with different layer was  employed and the result show that the networks with 8 neurons in  hidden layer has good agreement with experimental data.

 

Keywords: Zinc extraction, Efficiency, Neural networks, Operating condition.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 1542

Authors: S. H. Oudjana, A. Hellal, I. Hadj Mahammed

Abstract:

Photovoltaic power generation forecasting is an important task in renewable energy power system planning and operating. This paper explores the application of neural networks (NN) to study the design of photovoltaic power generation forecasting systems for one week ahead using weather databases include the global irradiance, and temperature of Ghardaia city (south of Algeria) using a data acquisition system. Simulations were run and the results are discussed showing that neural networks Technique is capable to decrease the photovoltaic power generation forecasting error.

Keywords: Photovoltaic Power Forecasting, Regression, Neural Networks.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 3701

Authors: Paul Lajbcygier, Seng Lee

Abstract:

Co-integration models the long-term, equilibrium relationship of two or more related financial variables. Even if cointegration is found, in the short run, there may be deviations from the long run equilibrium relationship. The aim of this work is to forecast these deviations using neural networks and create a trading strategy based on them. A case study is used: co-integration residuals from Australian Bank Bill futures are forecast and traded using various exogenous input variables combined with neural networks. The choice of the optimal exogenous input variables chosen for each neural network, undertaken in previous work [1], is validated by comparing the forecasts and corresponding profitability of each, using a trading strategy.

Keywords: Artificial neural networks, co-integration, forecasting, trading rule.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 1204

Authors: Qingqing Wang, Shouming Zhong

Abstract:

This paper deals with the problem of delay-dependent stability for neural networks with distributed delays. Some new sufficient condition are derived by constructing a novel Lyapunov-Krasovskii functional approach. The criteria are formulated in terms of a set of linear matrix inequalities, this is convenient for numerically checking the system stability using the powerful MATLAB LMI Toolbox. Moreover, in order to show the stability condition in this paper gives much less conservative results than those in the literature, numerical examples are considered.

Keywords: Neural networks, Globally asymptotic stability , LMI approach, Distributed delays.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 1521

Authors: Mohsen Hayati, Yazdan Shirvany

Abstract:

In this paper, the application of neural networks to study the design of short-term load forecasting (STLF) Systems for Illam state located in west of Iran was explored. One important architecture of neural networks named Multi-Layer Perceptron (MLP) to model STLF systems was used. Our study based on MLP was trained and tested using three years (2004-2006) data. The results show that MLP network has the minimum forecasting error and can be considered as a good method to model the STLF systems.

Keywords: Artificial neural networks, Forecasting, Multi-layer perceptron.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 2702

Authors: Z. Shaaban

Abstract:

This paper presents a new approach to tackle the problem of recognizing machine-printed Arabic texts. Because of the difficulty of recognizing cursive Arabic words, the text has to be normalized and segmented to be ready for the recognition stage. The new scheme for recognizing Arabic characters depends on multiple parallel neural networks classifier. The classifier has two phases. The first phase categories the input character into one of eight groups. The second phase classifies the character into one of the Arabic character classes in the group. The system achieved high recognition rate.

Keywords: Neural Networks, character recognition, feature extraction, multiple networks, Arabic text.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 1436

Authors: Zixin Liu, Huawei Yang, Fangwei Chen

Abstract:

In this paper, the issue of pth moment stability of a class of stochastic neural networks with mixed delays is investigated. By establishing two integro-differential inequalities, some new sufficient conditions ensuring pth moment exponential stability are obtained. Compared with some previous publications, our results generalize some earlier works reported in the literature, and remove some strict constraints of time delays and kernel functions. Two numerical examples are presented to illustrate the validity of the main results.

Keywords: Neural networks, stochastic, PTH moment stable, time varying delays, distributed delays.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 1426

Authors: Zixin Liu, Shu Lü, Shouming Zhong, Mao Ye

Abstract:

The problem of exponential stability and periodicity for a class of cellular neural networks (DCNNs) with time-varying delays is investigated. By dividing the network state variables into subgroups according to the characters of the neural networks, some sufficient conditions for exponential stability and periodicity are derived via the methods of variation parameters and inequality techniques. These conditions are represented by some blocks of the interconnection matrices. Compared with some previous methods, the method used in this paper does not resort to any Lyapunov function, and the results derived in this paper improve and generalize some earlier criteria established in the literature cited therein. Two examples are discussed to illustrate the main results.

Keywords: Cellular neural networks, exponential stability, time varying delays, partitioned matrices, periodic solution.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 1473

Authors: Rohan Putatunda, Aryya Gangopadhyay

Abstract:

Acoustic sensors, if embedded in smart street lights, can help in capturing the activities (car honking, sirens, events, traffic, etc.) in cities. Needless to say, the acoustic data from such scenarios are complex due to multiple audio streams originating from different events, and when decomposed to independent signals, the amount of retrieved data volume is small in quantity which is inadequate to train deep neural networks. So, in this paper, we address the two challenges: a) separating the mixed signals, and b) developing an efficient acoustic classifier under data paucity. So, to address these challenges, we propose an architecture with supervised deep learning, where the initial captured mixed acoustics data are analyzed with Fast Fourier Transformation (FFT), followed by filtering the noise from the signal, and then decomposed to independent signals by fast independent component analysis (Fast ICA). To address the challenge of data paucity, we propose a multi feature-based deep neural network with high performance that is reflected in our experiments when compared to the conventional convolutional neural network (CNN) and multi-layer perceptron (MLP).

Keywords: FFT, ICA, vehicle classification, multi-feature DNN, CNN, MLP.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 360

Authors: Apoorva Vinod, Archana Mathur, Snehanshu Saha

Abstract:

Though there exists a plethora of Activation Functions (AFs) used in single and multiple hidden layer Neural Networks (NN), their behavior always raised curiosity, whether used in combination or singly. The popular AFs – Sigmoid, ReLU, and Tanh – have performed prominently well for shallow and deep architectures. Most of the time, AFs are used singly in multi-layered NN, and, to the best of our knowledge, their performance is never studied and analyzed deeply when used in combination. In this manuscript, we experiment on multi-layered NN architecture (both on shallow and deep architectures; Convolutional NN and VGG16) and investigate how well the network responds to using two different AFs (Sigmoid-Tanh, Tanh-ReLU, ReLU-Sigmoid) used alternately against a traditional, single (Sigmoid-Sigmoid, Tanh-Tanh, ReLU-ReLU) combination. Our results show that on using two different AFs, the network achieves better accuracy, substantially lower loss, and faster convergence on 4 computer vision (CV) and 15 Non-CV (NCV) datasets. When using different AFs, not only was the accuracy greater by 6-7%, but we also accomplished convergence twice as fast. We present a case study to investigate the probability of networks suffering vanishing and exploding gradients when using two different AFs. Additionally, we theoretically showed that a composition of two or more AFs satisfies Universal Approximation Theorem (UAT).

Keywords: Activation Function, Universal Approximation function, Neural Networks, convergence.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 87

Authors: Adnene Arbi, Chaouki Aouiti, Abderrahmane Touati

Abstract:

In this paper, we consider the uniform asymptotic stability, global asymptotic stability and global exponential stability of the equilibrium point of discrete Hopfield neural networks with delays. Some new stability criteria for system are derived by using the Lyapunov functional method and the linear matrix inequality approach, for estimating the upper bound of Lyapunov functional derivative.

Keywords: Hopfield neural networks, uniform asymptotic stability, global asymptotic stability, exponential stability.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 1914

Authors: Tharini N. de Silva, Xiao Zhibo, Zhao Rui, Mao Kezhi

Abstract:

Causal relation identification is a crucial task in information extraction and knowledge discovery. In this work, we present two approaches to causal relation identification. The first is a classification model trained on a set of knowledge-based features. The second is a deep learning based approach training a model using convolutional neural networks to classify causal relations. We experiment with several different convolutional neural networks (CNN) models based on previous work on relation extraction as well as our own research. Our models are able to identify both explicit and implicit causal relations as well as the direction of the causal relation. The results of our experiments show a higher accuracy than previously achieved for causal relation identification tasks.

Keywords: Causal relation identification, convolutional neural networks, natural Language Processing, Machine Learning.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 2188

Authors: H. Sadjadian , H.D. Taghirad Member, A. Fatehi

Abstract:

In this paper, different approaches to solve the forward kinematics of a three DOF actuator redundant hydraulic parallel manipulator are presented. On the contrary to series manipulators, the forward kinematic map of parallel manipulators involves highly coupled nonlinear equations, which are almost impossible to solve analytically. The proposed methods are using neural networks identification with different structures to solve the problem. The accuracy of the results of each method is analyzed in detail and the advantages and the disadvantages of them in computing the forward kinematic map of the given mechanism is discussed in detail. It is concluded that ANFIS presents the best performance compared to MLP, RBF and PNN networks in this particular application.

Keywords: Forward Kinematics, Neural Networks, Numerical Solution, Parallel Manipulators.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 1880

Authors: S. Mousavian, A. Abedianpour, A. Khanmohammadi, S. Hematian, Gh. Eidi Veisi

Abstract:

Ionic liquids are finding a wide range of applications from reaction media to separations and materials processing. In these applications, Vapor–Liquid equilibrium (VLE) is the most important one. VLE for six systems at 353 K and activity coefficients at infinite dilution [(γ)_i^∞] for various solutes (alkanes, alkenes, cycloalkanes, cycloalkenes, aromatics, alcohols, ketones, esters, ethers, and water) in the ionic liquids (1-ethyl-3-methylimidazolium bis (trifluoromethylsulfonyl)imide [EMIM][BTI], 1-hexyl-3-methyl imidazolium bis (trifluoromethylsulfonyl) imide [HMIM][BTI], 1-octyl-3-methylimidazolium bis(trifluoromethylsulfonyl) imide [OMIM][BTI], and 1-butyl-1-methylpyrrolidinium bis (trifluoromethylsulfonyl) imide [BMPYR][BTI]) have been used to train neural networks in the temperature range from (303 to 333) K. Densities of the ionic liquids, Hildebrant constant of substances, and temperature were selected as input of neural networks. The networks with different hidden layers were examined. Networks with seven neurons in one hidden layer have minimum error and good agreement with experimental data.

Keywords: Ionic liquid, Neural networks, VLE, Dilute solution.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 1320

Authors: Defu Zhang, Qingshan Jiang, Xin Li

Abstract:

This paper deals with the application of a well-known neural network technique, multilayer back-propagation (BP) neural network, in financial data mining. A modified neural network forecasting model is presented, and an intelligent mining system is developed. The system can forecast the buying and selling signs according to the prediction of future trends to stock market, and provide decision-making for stock investors. The simulation result of seven years to Shanghai Composite Index shows that the return achieved by this mining system is about three times as large as that achieved by the buy and hold strategy, so it is advantageous to apply neural networks to forecast financial time series, the different investors could benefit from it.

Keywords: Data mining, neural network, stock forecasting.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 3539

Authors: Ozlem Faydasicok, Sabri Arik

Abstract:

This paper derives some new sufficient conditions for the stability of a class of neutral-type neural networks with discrete time delays by employing a suitable Lyapunov functional. The obtained conditions can be easily verified as they can be expressed in terms of the network parameters only. It is shown that the results presented in this paper for neutral-type delayed neural networks establish a new set of stability criteria, and therefore can be considered as the alternative results to the previously published literature results. A numerical example is also given to demonstrate the applicability of our proposed stability criterion.

Keywords: Stability Analysis, Neutral-Type Neural Networks, Time Delay Systems, Lyapunov Functionals.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 1596

Authors: Zixin Liu, Shu Lü, Shouming Zhong, Mao Ye

Abstract:

This paper studies the pth moment exponential synchronization of a class of stochastic neural networks with mixed delays. Based on Lyapunov stability theory, by establishing a new integrodifferential inequality with mixed delays, several sufficient conditions have been derived to ensure the pth moment exponential stability for the error system. The criteria extend and improve some earlier results. One numerical example is presented to illustrate the validity of the main results.

Keywords: pth Moment Exponential synchronization, Stochastic, Neural networks, Mixed time delays

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 1538

Authors: Hossein Sahoolizadeh, Mahdi Rahimi, Hamid Dehghani

Abstract:

We introduce an algorithm based on the morphological shared-weight neural network. Being nonlinear and translation-invariant, the MSNN can be used to create better generalization during face recognition. Feature extraction is performed on grayscale images using hit-miss transforms that are independent of gray-level shifts. The output is then learned by interacting with the classification process. The feature extraction and classification networks are trained together, allowing the MSNN to simultaneously learn feature extraction and classification for a face. For evaluation, we test for robustness under variations in gray levels and noise while varying the network-s configuration to optimize recognition efficiency and processing time. Results show that the MSNN performs better for grayscale image pattern classification than ordinary neural networks.

Keywords: Face recognition, Neural Networks, Multi-layer Perceptron, masking.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 1472

Authors: L. Parisi

Abstract:

In this research work, neural networks were applied to classify two types of hip joint implants based on the relative hip joint implant side speed and three components of each ground reaction force. The condition of walking gait at normal velocity was used and carried out with each of the two hip joint implants assessed. Ground reaction forces’ kinetic temporal changes were considered in the first approach followed but discarded in the second one. Ground reaction force components were obtained from eighteen patients under such gait condition, half of which had a hip implant type I-II, whilst the other half had the hip implant, defined as type III by Orthoload®. After pre-processing raw gait kinetic data and selecting the time frames needed for the analysis, the ground reaction force components were used to train a MLP neural network, which learnt to distinguish the two hip joint implants in the abovementioned condition. Further to training, unknown hip implant side and ground reaction force components were presented to the neural networks, which assigned those features into the right class with a reasonably high accuracy for the hip implant type I-II and the type III. The results suggest that neural networks could be successfully applied in the performance assessment of hip joint implants.

Keywords: Kinemic gait data, Neural networks, Hip joint implant, Hip arthroplasty, Rehabilitation Engineering.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 1748

Authors: M. Tarafdar Haque, A. Taheri

Abstract:

Application of neural networks in execution of programmed pulse width modulation (PPWM) of a voltage source inverter (VSI) is studied in this paper. Using the proposed method it is possible to cancel out the desired harmonics in output of VSI in addition to control the magnitude of fundamental harmonic, contineously. By checking the non-trained values and a performance index, the most appropriate neural network is proposed. It is shown that neural networks may solve the custom difficulties of practical utilization of PPWM such as large size of memory, complex digital circuits and controlling the magnitude of output voltage in a discrete manner.

Keywords: Neural Network, Inverter, PPWM.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 1643

Authors: Miaomiao Yang, Shouming Zhong

Abstract:

This paper studies the problem of exponential stability analysis for recurrent neural networks with time-varying delay.By establishing a suitable augmented LyapunovCKrasovskii function and a novel sufficient condition is obtained to guarantee the exponential stability of the considered system.In order to get a less conservative results of the condition,zero equalities and reciprocally convex approach are employed. The several exponential stability criterion proposed in this paper is simpler and effective. A numerical example is provided to demonstrate the feasibility and effectiveness of our results.

Keywords: Exponential stability , Neural networks, Linear matrix inequality, Lyapunov-Krasovskii, Time-varying.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 1710

Authors: Danilo López, Nelson Vera, Luis Pedraza

Abstract:

This paper analyzes fundamental ideas and concepts related to neural networks, which provide the reader a theoretical explanation of Long Short-Term Memory (LSTM) networks operation classified as Deep Learning Systems, and to explicitly present the mathematical development of Backward Pass equations of the LSTM network model. This mathematical modeling associated with software development will provide the necessary tools to develop an intelligent system capable of predicting the behavior of licensed users in wireless cognitive radio networks.

Keywords: Neural networks, multilayer perceptron, long short-term memory, recurrent neuronal network, mathematical analysis.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 1511

Authors: Vishnu Pratap Singh Kirar

Abstract:

Artificial Neural Network (ANN) can be trained using back propagation (BP). It is the most widely used algorithm for supervised learning with multi-layered feed-forward networks. Efficient learning by the BP algorithm is required for many practical applications. The BP algorithm calculates the weight changes of artificial neural networks, and a common approach is to use a twoterm algorithm consisting of a learning rate (LR) and a momentum factor (MF). The major drawbacks of the two-term BP learning algorithm are the problems of local minima and slow convergence speeds, which limit the scope for real-time applications. Recently the addition of an extra term, called a proportional factor (PF), to the two-term BP algorithm was proposed. The third increases the speed of the BP algorithm. However, the PF term also reduces the convergence of the BP algorithm, and criteria for evaluating convergence are required to facilitate the application of the three terms BP algorithm. Although these two seem to be closely related, as described later, we summarize various improvements to overcome the drawbacks. Here we compare the different methods of convergence of the new three-term BP algorithm.

Keywords: Neural Network, Backpropagation, Local Minima, Fast Convergence Rate.

Procedia APA BibTeX Chicago EndNote Harvard JSON MLA RIS XML ISO 690 PDF Downloads 3499