Adaptive PID Controller based on Reinforcement Learning for Wind Turbine Control

Authors: M. Sedighizadeh, A. Rezazadeh

Abstract:

A self tuning PID control strategy using reinforcement learning is proposed in this paper to deal with the control of wind energy conversion systems (WECS). Actor-Critic learning is used to tune PID parameters in an adaptive way by taking advantage of the model-free and on-line learning properties of reinforcement learning effectively. In order to reduce the demand of storage space and to improve the learning efficiency, a single RBF neural network is used to approximate the policy function of Actor and the value function of Critic simultaneously. The inputs of RBF network are the system error, as well as the first and the second-order differences of error. The Actor can realize the mapping from the system state to PID parameters, while the Critic evaluates the outputs of the Actor and produces TD error. Based on TD error performance index and gradient descent method, the updating rules of RBF kernel function and network weights were given. Simulation results show that the proposed controller is efficient for WECS and it is perfectly adaptable and strongly robust, which is better than that of a conventional PID controller.

Keywords: Wind energy conversion systems, reinforcementlearning; Actor-Critic learning; adaptive PID control; RBF network.

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1057789

References:

[1] Kanellos, F.D., Hatziargyriou, N.D., 2002. A new control scheme for variable speed wind turbine using neural networks. IEEE Power Engineering Society Winter Meeting, 1:
[2] You-tong, F., Cheng-zhi, F., 2007. Single neuron network PI control of high reliability linear induction motor for Maglev. Journal of Zhejiang University SCIENCE A, 2007, 8(3):408-411.
[3] Zhao-da, Y., Chong-guang, Z., Shi-chuan, S., Zhen-tao, L., Xi-zhen, W., 2003. Application of neural network in the study of combustion rate of natural gas/diesel dual fuel engine. Journal of Zhejiang University SCIENCE A, 2003, 4(2):170-174
[4] Haykin, S., 1994. Neural Networks, A Comprehensive Foundation. New York: Macmillan, 1994.
[5] Mayosky, M. A., Cancelo, G. I. E., 1999. Direct adaptive control of wind energy conversion systems using gaussian networks. IEEE Transactions on neural networks, 10(4): 898-906.
[6] Kalantar, M., Sedighizadeh, M., 2004. Adaptive Self Tuning Control of Wind Energy Conversion Systems Using Morlet Mother Wavelet Basis Functions Networks. 12th Mediterranean IEEE Conference on Control and Automation MED-04 , Kusadasi, Turkey.
[7] Sedighizadeh, M., Kalantar, M., 2004. Adaptive PID Control of Wind Energy Conversion Systems Using RASP1 Mother Wavelet Basis Function Networks. IEEE TENCON 2004, Chiang Mai, Thailand.
[8] Sedighizadeh, M., et al, 2005. Nonlinear Model Identification and Control of Wind Turbine Using Wavenets. Proceedings of the 2005 IEEE Conference on Control Applications Toronto, Canada, PP.1057- 1062.
[9] WANG Xue-song, CHENG Yu-hu, SUN Wei. A Proposal of Adaptive PID Controller Based on Reinforcement LearningJ China Univ Mining & Technol 2007, 17(1): 0040-0044.
[10] Wang X S, Cheng Y H, Sun W. Q learning based on self-organizing fuzzy radial basis function network. Lecture Notes inComputer Science, 2006, 3971: 607-615.
[11] Barto A G, Sutton R S, Anderson C W. Neuronlike adaptive elements that can solve difficult learning control problems. IEEETransactions on Systems, Man and Cybernetics, 1983, 13(5): 834-846.