Reinforcement Learning-Based Coexistence Interference Management in Wireless Body Area Networks

Authors: Izaz Ahmad, Farhatullah, Shahbaz Ali, Farhad Ali, Faiza, Hazrat Junaid, Farhan Zaid

Abstract:

Current trends in remote health monitoring to monetize on the Internet of Things applications have been raised in efficient and interference free communications in Wireless Body Area Network (WBAN) scenario. Co-existence interference in WBANs have aggravates the over-congested radio bands, thereby requiring efficient Carrier Sense Multiple Access with Collision Avoidance (CSMA/CA) strategies and improve interference management. Existing solutions utilize simplistic heuristics to approach interference problems. The scope of this research article is to investigate reinforcement learning for efficient interference management under co-existing scenarios with an emphasis on homogenous interferences. The aim of this paper is to suggest a smart CSMA/CA mechanism based on reinforcement learning called QIM-MAC that effectively uses sense slots with minimal interference. Simulation results are analyzed based on scenarios which show that the proposed approach maximized Average Network Throughput and Packet Delivery Ratio and minimized Packet Loss Ratio, Energy Consumption and Average Delay.

Keywords: WBAN, IEEE 802.15.4 Standard, CAP Super-frame, Q-Learning.

References:

[1] A. Astrin, "IEEE Standard for Local and Metropolitan Area Networks, Wireless Body Area Networks," IEEE, 29 Feb 2012.
[2] B. B. Benoit Latrie, Ingrid Moerman, Chris Blondia, and Piet Demeester, "A survey on wireless body area networks," Wireless Networks, pp. pp 1-18, 2010.
[3] J.-H. C. Beomseok Kim, and Jinsung Cho, Member, "A Hybrid Channel Access Scheme for Coexistence Mitigation in IEEE 802.15.4-Based WBAN," IEEE, vol. Vol. 17, NO. 21, 1 November 2017.
[4] R. M. Cavallari, F. Rosini, R. Buratti, C. Verdone, "A Survey on Wireless Body Area Networks: Technologies and Design Challenges," IEEE Communication Survey, 2014.
[5] G. T. Chen, Chen, W.T, Shen, "2L-MAC: A MAC Protocol with Two-Layer Interference Mitigation in Wireless Body Area Networks for Medical Applications in Proceeding of the IEEE International Conference on Communications (ICC), Sydney, Australia," June 2014.
[6] M. N. J. Deylami, "E. A Distributed Scheme to Manage the Dynamic Coexistence of IEEE 802.15.4-Based Health-Monitoring WBANs," IEEE, 2013.
[7] M. M. Fumihiro Inoue, Takayuki Nishio, Koji Yamamoto, Fusao Nuno, Takatoshi Sugiyama, "Novel coexistence scheme between wireless sensor network and wireless LAN for HEMS. Smart Grid Communication," IEEE, pp. pp 271-276, 2013.
[8] A. M. Jamthe, A. Agrawal,, "Scheduling Schemes for Interference Suppression in Healthcare Sensor Networks," In Proceeding of the IEEE International Conference on Communications (ICC), Sydney, Australia, June 2014.
[9] L. P. Kaelbling, Littman, M. L, "Reinforcement learning: A survey. Journal of Artificial Intelligence Research," pp. pp 237 - 285,, 1996.
[10] B. B. Latrfe, B. Moerman, I. Blondia, C. De-meester, "A survey on wireless body area networks. Wireless, Network," IEEE, 17 Jan 2011.
[11] J. M. Mahapatro, S. Manjunatha, M. Islam, "Interference-Aware Channel Switching for Use in WBAN with Human-Sensor Interface," n Proceeding of the 4th International Conference on Intelligent Human Computer Interaction (IHCI), Kharagpur, India, December 2012.
[12] H. M. Mohamad Jaafar Ali, Ahmed Mehaoua, "Interference Avoidance Algorithm (IAA) for Multi-hop Wireless Body Area Network Communication" arXiv:1602.08658v2 (cs.NI), 2 November 2016.
[13] S. A. Movassaghi, M. Lipman, J. Smith, D. Jamalipour, "A Wireless Body Area Networks: A Survey," IEEE, 2014.
[14] S. A. Movassaghi, M. Smith, "Smart Spectrum Allocation for Interference Mitigation in Wireless Body Area Networks in Proceeding of the IEEE International Conference on Communications (ICC), Sydney, Australia," IEEE, June 2014.
[15] C. M. Otto, A. Sanders, C. Jovanov, "System Architecture of a Wireless Body Area Sensor Network for Ubiquitous Health Monitoring," pp. pp 307 - 326, 2006.
[16] M. W. A. Patel, "Challenges, and Prospective in Emerging Body Area Networking Technologies," IEEE, pp. pp 80-88, 2010.
[17] T. J. Ratko Magjarevic, Peter Kramar, and Anze Zupanic, Home care technologies for ambient assisted living, in 11th Mediterranean Conference on Medical and Biomedical Engineering and Computing. 2007.
[18] A. M. Samaneh Movassaghii, Abbas Jamalipour, David Smith, Mehran Abolhasan, "Enabling Interference-Aware and Energy-Efficient Coexistence of Multiple Wireless Body Area Networks with Unknown Dynamics. Digital Object Identifier 10.1109/ACCESS," 2016.
[19] I. Standard, "Wireless Medium Access Control (MAC) and Physical Layer (PHY) Specifications for Low-Rate Wireless Personal Area Networks (LR - WPANs)," IEEE, 2006.
[20] R. S. Sutton, and Barto, "Reinforcement learning: An introduction. MIT Press," 1998.
[21] C. J. Watkins, Dayan, "P. Q-learning. Machine Learning," pp. pp 279 - 292, 1992.
[22] C. J. C. H. Watkins, "Learning from delayed rewards. PhD thesis, University of Cambridge, England," 1989.
[23] A.E. Forster, A.L. Murphy “FORMS: Feedbacks Routing for Optimization Multiple Sinks in WSN with Reinforcement Learning” in 3rd International Conference on Intelligent Sensors, Sensor Networks and Information. Melbourne, Qld., Australia. IEEE, dec 3-6. 2007
[24] A. Förster and A. Murphy, "Froms: A failure tolerant and mobility enabled multicast routing paradigm with reinforcement learning for WSNs", Ad Hoc Networks, vol. 9, no. 5, pp. 940-965, 2011. Available: 10.1016/j.adhoc.2010.11.006.