Periodic Storage Control Problem

Authors: Ru-Shuo Sheu, Han-Hsin Chou, Te-Shyang Tan

Abstract:

Considering a reservoir with periodic states and different cost functions with penalty, its release rules can be modeled as a periodic Markov decision process (PMDP). First, we prove that policy- iteration algorithm also works for the PMDP. Then, with policy- iteration algorithm, we obtain the optimal policies for a special aperiodic reservoir model with two cost functions under large penalty and give a discussion when the penalty is small.

Keywords: periodic Markov decision process, periodic state, policy-iteration algorithm.

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1330529

References:

[1] Goulter, I.C. and Tai, F-K.(1985) Practical Implications in the Use of Stochastic Dynamic Programing for Optimal Reservoir Operation. Water Resources Bulletin 21, No.1 65-74.
[2] Heidari, M., Chow, V.T., Kokotovic, P.V.and Meredith, D.D.(1971) Discrete Differential Dynamic Programing Approach to Water Resources System Optimization. Water Resources Research 7, No.2 273-282.
[3] Howard, R.A. Dynamic Programming and Markov Processes MIT Press, Cambridge, Mass.(1960).
[4] Moran, P.A.P.(1954) Probability Theory of Dams and Storage Systems. Aust. J. Appl. Sci. 5 116-124.
[5] Moran, P.A.P.(1955) A Probability Theory of Dams and Storage Systems: Modifications of the Release Rules. Aust. J. Appl. Sci. 6 117-130.
[6] Nagy, I.V. ,Asante-Duah, K. and Zsuffa, I., Hydrological Dimensioning and Operation of Reservoirs Kluwer Academic Publishers(2002).
[7] Nopmongcol, P. and Askew, A.J.(1976) Multilevel Incremental Dynamic Programing. Water Resources Research 12, No.6 1291-1297.
[8] Reznicek, K. and Cheng, T.C.E.(1991) Stochastic Modelling of Reservoir Operations. European Journal of Operation Research 50 235-248.
[9] Ru-Shuo Sheu, Pang-Ju Lou, and Han-Hsin Chou(2011) Reservoir Storage Release Rules Control Problem. 2011 International Symposium on Water Resource and Environmental Protection (ISWREP 2011)(IEEE)
[10] Tijms, H.C. Stochastic Models : An Algorithmic Approach John Wiley & Sons, Inc. New York, (1994).
[11] Turgeon, A. (1980) Optimal Operation of Multireservoir Power System with Stochastic Inflows. Water Resources Research 16, No.2 275-283.
[12] Wurbs, R.A. (1993) Reservoir-System Simulation and Optimization Models. Journal of Water Resources Planning and Management 119, No.4 455-472.
[13] Yeh, W.W-G.(1985) Reservoir Management and Operations Models : A State-of-Art Review. Water Resources Research 21, No.12 1797-1818.