Agent-based Simulation for Blood Glucose Control in Diabetic Patients

Authors: Sh. Yasini, M. B. Naghibi-Sistani, A. Karimpour

Abstract:

This paper employs a new approach to regulate the blood glucose level of type I diabetic patient under an intensive insulin treatment. The closed-loop control scheme incorporates expert knowledge about treatment by using reinforcement learning theory to maintain the normoglycemic average of 80 mg/dl and the normal condition for free plasma insulin concentration in severe initial state. The insulin delivery rate is obtained off-line by using Qlearning algorithm, without requiring an explicit model of the environment dynamics. The implementation of the insulin delivery rate, therefore, requires simple function evaluation and minimal online computations. Controller performance is assessed in terms of its ability to reject the effect of meal disturbance and to overcome the variability in the glucose-insulin dynamics from patient to patient. Computer simulations are used to evaluate the effectiveness of the proposed technique and to show its superiority in controlling hyperglycemia over other existing algorithms

Keywords: Insulin Delivery rate, Q-learning algorithm, Reinforcement learning, Type I diabetes.

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1060215

References:

[1] American Diabetes Association, "Standards of medical care for patients with diabetes mellitus," Diabetes Care, vol. 26, pp. S33-S50, 2003.
[2] B. Topp, K. Promislow, G. De Vries, "A model of β-cell mass, insulin, and glucose kinetics: Pathway to diabetes," Journal of Theoretical Biology. Vol. 206, 2000, pp. 6.5-619.
[3] "The effect of intensive treatment of diabetes on the development and progression of long-term complications in insulin-dependent diabetes mellitus," N. Eng. J. Med, vol. 329, pp. 977-986, 1993, DCCT.
[4] A. Albisser, B. Leibel, T. Ewartz, Z. Davidovac, C. Botz, W. H. Zingg," artificial endocrine pancreas," Diabetes Care, vol. 23, 1974, pp. 389- 396.
[5] J. Jaremco, O. Rorstad, "Advances toward implantable artificial pancreas for treatment of diabetes," Diabetes Care, vol. 21, 1998, pp. 444-450.
[6] M. Nalecz, J. Wojcicki, and I. Zawicki, "Control in artificial pancreas," IFAC Control Aspects Biomed Eng, 1987, pp. 123-137.
[7] R. S. parker, G. L. Bowlin, and G. Wnek, Eds. "Insulin delivery," Encyclopedia of Biomaterials and Biomedical Engineering. New Yourk: Market Dekker, 2004, pp. 857-866.
[8] F. P. Kennedy, "Recent development in insulin deliver technique: current status or future potential," Drugs, vol. 42, 1991, pp. 213-227.
[9] DC. Klonoff. "Continuous glucose monitoring: roadmap for 21st. century diabetes therapy," Diabetes care Vol. 28, 2005, pp. 1231-1239.
[10] C. Amaral, B. Wolf, "Current development in non-invasive glucose monitoring," Journal of Medical Engineering and Physics, 2007, Elsevier.
[11] TM. Gross, BW bode, D. Einhorn, "Performance evaluation of the minimed continuous glucose monitoring system during patient home use," Diabetes Technology Theoretical. vol. 2, 2000, pp. 49-56.
[12] J. Li, Y. Kuang, C. C. Mason, "Modeling the glucose-insulin regulatory system and ultradian insulin secretory oscillations with two explicit time delays," J. of Theoretical Biology, vol. 242, 2006, pp. 722-735.
[13] A. De Gaetano and O. Arino, "Mathematical modeling of the intravenous glucose tolerance test", J. Math. Biol. Vol. 40, 2000, pp. 136-168.
[14] R. N. Bergman, L. Philips, and C. Cobelli, "Physiological evaluation of the factors controlling glucose tolerance in man," Journal of Clinical Investigation, vol. 68, 1981, pp. 1456-1467.
[15] J. T. Sorenson, "A physiological model of glucose metabolism in mans its use to design and assess improved insulin therapies for diabetes," ", Ph.D. Dissertation, Chem Eng. Dep., Massachusets Inst. Technol, Cambridge, 1985
[16] Ch. Li and R. Hu, "Simulation study on blood glucose control in diabetics", Proc. IEEE Int. Conf. on Biomed. and Bioinf Eng, 2007, pp. 1103-1106.
[17] F. Chee, T. L. Fernando, A. V. Savkin, and V. Heeden, "Expert PID control system for blood glucose control on critically ill patients," IEEE Trans. Biomed. Eng., vol. 7, No. 4, 2003, pp. 419-425.
[18] Z. H. Lam, J. Y. Hwang, J. G. Lee, J. G. Chase, and G. C. Wake, "Active insulin infusion using optimal and derivative-weighted control," Medical Engineering & Physics. vol. 24, 2002, pp. 663-672.
[19] K. H. Kientiz and T. Yoneyame, "A robust controller for insulin pupms based on H-infinity theory" IEEE Transaction on Biomedical Eng., vol. 40, 1993, pp. 1133-1137.
[20] Sh. Yasini, A. Karimpour, M. B. Naghibi-Sistani, S. Ghareh., "An automatic insulin infusion system based on H-infinity control technique," Procceding of the 2008 IEEE, CIBEC-08, 2008, pp. 1-5, Cairo, Egypt.
[21] F. Chee, AV. Savkin, TL Fernando, S. Nahavandi, "optimal H-infinity insulin injection control for blood glucose regulation in diabetic patients," vol. 52, No. 10, 2005, pp. 1625-1631.
[22] M. S. Ibbini, M. A. Masadeh, and M. M. Bani Amer, "A semi closed loop optimal control system blood glucose level in diabetics", J. Medical Eng. & Tech. Vol. 28., 2004, pp. 189-196.
[23] M. E. Fisher, "A semi closed-loop algorithm for control of blood glucose levels in diabetics", IEEE Trans. On Biomed. Eng, vol.38, 38. No. 1, 1991.
[24] R. S. Sutton, A. G. Barto, Reinforcement learning: An introduction, 1989, MIT Press.
[25] R. N. Bergman, D. T. Finegood, M. Ader, "Assessment of insulin sensitivity invivo", Endocrine Reviews, Vol. 6, No. 1, 1985, pp. 45-85.
[26] C. Neatpisarnvanit, JR Boston, "Estimation of plasma insulin for plasma glucose. IEEE Transaction on Biomedical Engineering," vol. 49, No. 11, 2002, pp.1253-1259.
[27] C. Watkins, Learning from delayed rewards, Ph.D. Dissertation Cambridge University, 1998.