Optimal Maintenance and Improvement Policies in Water Distribution System: Markov Decision Process Approach

Authors: Jong Woo Kim, Go Bong Choi, Sang Hwan Son, Dae Shik Kim, Jung Chul Suh, Jong Min Lee

Abstract:

The Markov decision process (MDP) based methodology is implemented in order to establish the optimal schedule which minimizes the cost. Formulation of MDP problem is presented using the information about the current state of pipe, improvement cost, failure cost and pipe deterioration model. The objective function and detailed algorithm of dynamic programming (DP) are modified due to the difficulty of implementing the conventional DP approaches. The optimal schedule derived from suggested model is compared to several policies via Monte Carlo simulation. Validity of the solution and improvement in computational time are proved.

Keywords: Markov decision processes, Dynamic Programming, Monte Carlo simulation, Periodic replacement, Weibull distribution.

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1100208

References:

[1] K. Boston and P. Bettinger, “An analysis of monte carlo integer programming, simulated annealing, and tabu search heuristics for solving spatial harvest scheduling problems,” Forest Science, vol. 45, no. 2, pp. 292–301, 1999.
[2] S. A. Andreou, D. H. Marks, and R. M. Clark, “A new methodology for modelling break failure patterns in deteriorating water distribution systems: Applications,” Advances in Water Resources, vol. 10, no. 1, pp. 11–20, 1987.
[3] Y. Le Gat and P. Eisenbeis, “Using maintenance records to forecast failures in water networks,” Urban Water, vol. 2, no. 3, pp. 173–181, 2000.
[4] D. Li and Y. Y. Haimes, “Optimal maintenance-related decision making for deteriorating water distribution systems: 1. semi-markovian model for a water main,” Water Resources Research, vol. 28, no. 4, pp. 1053–1061, 1992.
[5] M. A. Cesare, C. Santamarina, C. Turkstra, and E. H. Vanmarcke, “Modeling bridge deterioration with markov chains,” Journal of Transportation Engineering, vol. 118, no. 6, pp. 820–833, 1992.
[6] Y. Kleiner, “Scheduling inspection and renewal of large infrastructure assets,” Journal of Infrastructure Systems, vol. 7, no. 4, pp. 136–143, 2001.
[7] S. Madanat, R. Mishalani, and W. H. W. Ibrahim, “Estimation of infrastructure transition probabilities from condition rating data,” Journal of infrastructure systems, vol. 1, no. 2, pp. 120–125, 1995.
[8] H.-S. Baik, H. S. Jeong, and D. M. Abraham, “Estimating transition probabilities in markov chain-based deterioration models for management of wastewater systems,” Journal of water resources planning and management, vol. 132, no. 1, pp. 15–24, 2006.
[9] F. Guignier and S. Madanat, “Optimization of infrastructure systems maintenance and improvement policies,” Journal of Infrastructure Systems, vol. 5, no. 4, pp. 124–134, 1999.
[10] S. Madanat and W. H. W. Ibrahim, “Poisson regression models of infrastructure transition probabilities,” Journal of Transportation Engineering, vol. 121, no. 3, pp. 267–272, 1995.
[11] R. Wirahadikusumah, D. Abraham, and T. Iseley, “Challenging issues in modeling deterioration of combined sewers,” Journal of infrastructure systems, vol. 7, no. 2, pp. 77–84, 2001.
[12] T. Micevski, G. Kuczera, and P. Coombes, “Markov model for storm water pipe deterioration,” Journal of Infrastructure Systems, vol. 8, no. 2, pp. 49–56, 2002.
[13] G. C. Dandy and M. O. Engelhardt, “Multi-objective trade-offs between cost and reliability in the replacement of water mains,” Journal of water resources planning and management, vol. 132, no. 2, pp. 79–88, 2006.
[14] A. Nafi, C. Werey, and P. Llerena, “Water pipe renewal using a multiobjective optimization approach,” Canadian Journal of Civil Engineering, vol. 35, no. 1, pp. 87–94, 2008.
[15] G. V. Loganathan, S. Park, and H. Sherali, “Threshold break rate for pipeline replacement in water distribution systems,” Journal of water resources planning and management, vol. 128, no. 4, pp. 271–279, 2002.
[16] Y. Kleiner, B. J. Adams, and J. S. Rogers, “Long-term planning methodology for water distribution system rehabilitation,” Water resources research, vol. 34, no. 8, pp. 2039–2051, 1998.
[17] S. Madanat and M. Ben-Akiva, “Optimal inspection and repair policies for infrastructure facilities,” Transportation science, vol. 28, no. 1, pp. 55–62, 1994.
[18] G. Buxey, “The vehicle scheduling problem and monte carlo simulation,” Journal of the Operational Research Society, pp. 563–573, 1979.
[19] M. L. Puterman, Markov decision processes: discrete stochastic dynamic programming, vol. 414. John Wiley & Sons, 2009.
[20] V. Kathuls and R. McKim, “sewer deterioration prediction,” in Proc., Infra 99 Int. Convention, 1999.
[21] S. A. Andreou, D. H. Marks, and R. M. Clark, “A new methodology for modelling break failure patterns in deteriorating water distribution systems: Theory,” Advances in Water Resources, vol. 10, no. 1, pp. 2–10, 1987.