Discussing Embedded versus Central Machine Learning in Wireless Sensor Networks

Authors: Anne-Lena Kampen, Øivind Kure

Abstract:

Machine learning (ML) can be implemented in Wireless Sensor Networks (WSNs) as a central solution or distributed solution where the ML is embedded in the nodes. Embedding improves privacy and may reduce prediction delay. In addition, the number of transmissions is reduced. However, quality factors such as prediction accuracy, fault detection efficiency and coordinated control of the overall system suffer. Here, we discuss and highlight the trade-offs that should be considered when choosing between embedding and centralized ML, especially for multihop networks. In addition, we present estimations that demonstrate the energy trade-offs between embedded and centralized ML. Although the total network energy consumption is lower with central prediction, it makes the network more prone for partitioning due to the high forwarding load on the one-hop nodes. Moreover, the continuous improvements in the number of operations per joule for embedded devices will move the energy balance toward embedded prediction.

Keywords: Central ML, embedded machine learning, energy consumption, local ML, Wireless Sensor Networks, WSN.

References:

[1] X. Li, D. Li, J. Wan, A. V. Vasilakos, C.-F. Lai, and S. Wang, "A review of industrial wireless networks in the context of Industry 4.0," Wireless networks, vol. 23, no. 1, pp. 23-41, 2017.
[2] D.-H. Kim et al., "Smart machining process using machine learning: A review and perspective on machining industry," International Journal of Precision Engineering and Manufacturing-Green Technology, vol. 5, no. 4, pp. 555-568, 2018.
[3] W. Liu, Z. Wang, X. Liu, N. Zeng, Y. Liu, and F. E. Alsaadi, "A survey of deep neural network architectures and their applications," Neurocomputing, vol. 234, pp. 11-26, 2017.
[4] K. Vamsikrishna, D. P. Dogra, and M. S. Desarkar, "Computer-vision-assisted palm rehabilitation with supervised learning," IEEE Transactions on Biomedical Engineering, vol. 63, no. 5, pp. 991-1001, 2015.
[5] F. Tao and C. Busso, "Gating neural network for large vocabulary audiovisual speech recognition," IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 26, no. 7, pp. 1290-1302, 2018.
[6] R. Girshick, J. Donahue, T. Darrell, and J. Malik, "Rich feature hierarchies for accurate object detection and semantic segmentation," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2014, pp. 580-587.
[7] M. Abadi et al., "Tensorflow: Large-scale machine learning on heterogeneous distributed systems," arXiv preprint arXiv:1603.04467, 2016.
[8] L. Dekker and C. Ritsema, "Wetting patterns and moisture variability in water repellent Dutch soils," Journal of Hydrology, vol. 231, pp. 148-164, 2000.
[9] L. Meng and S. M. Quiring, "A comparison of soil moisture models using soil climate analysis network observations," Journal of Hydrometeorology, vol. 9, no. 4, pp. 641-659, 2008.
[10] N. D. Lane, S. Bhattacharya, A. Mathur, P. Georgiev, C. Forlivesi, and F. Kawsar, "Squeezing deep learning into mobile and embedded devices," IEEE Pervasive Computing, vol. 16, no. 3, pp. 82-88, 2017.
[11] M. Mohammadi, A. Al-Fuqaha, S. Sorour, and M. Guizani, "Deep learning for IoT big data and streaming analytics: A survey," IEEE Communications Surveys & Tutorials, vol. 20, no. 4, pp. 2923-2960, 2018.
[12] F. Samie, S. Paul, L. Bauer, and J. Henkel, "Highly efficient and accurate seizure prediction on constrained iot devices," in 2018 Design, Automation & Test in Europe Conference & Exhibition (DATE), 2018: IEEE, pp. 955-960.
[13] N. D. Lane, S. Bhattacharya, P. Georgiev, C. Forlivesi, and F. Kawsar, "An early resource characterization of deep learning on wearables, smartphones and internet-of-things devices," in Proceedings of the 2015 international workshop on internet of things towards applications, 2015, pp. 7-12.
[14] A. E. Eshratifar and M. Pedram, "Energy and performance efficient computation offloading for deep neural networks in a mobile cloud computing environment," in Proceedings of the 2018 on Great Lakes Symposium on VLSI, 2018, pp. 111-116.
[15] H. Li, K. Ota, and M. Dong, "Learning IoT in edge: Deep learning for the Internet of Things with edge computing," IEEE network, vol. 32, no. 1, pp. 96-101, 2018.
[16] J. Lee, M. Stanley, A. Spanias, and C. Tepedelenlioglu, "Integrating machine learning in embedded sensor systems for Internet-of-Things applications," in 2016 IEEE international symposium on signal processing and information technology (ISSPIT), 2016: IEEE, pp. 290-294.
[17] Y. Fukushima, D. Miura, T. Hamatani, H. Yamaguchi, and T. Higashino, "MicroDeep: In-network deep learning by micro-sensor coordination for pervasive computing," in 2018 IEEE International Conference on Smart Computing (SMARTCOMP), 2018: IEEE, pp. 163-170.
[18] X. Fafoutis, L. Marchegiani, A. Elsts, J. Pope, R. Piechocki, and I. Craddock, "Extending the battery lifetime of wearable sensors with embedded machine learning," in 2018 IEEE 4th World Forum on Internet of Things (WF-IoT), 2018: IEEE, pp. 269-274.
[19] P. Park, S. C. Ergen, C. Fischione, C. Lu, and K. H. Johansson, "Wireless network design for control systems: A survey," IEEE Communications Surveys & Tutorials, vol. 20, no. 2, pp. 978-1013, 2017.
[20] C. Zhang, P. Patras, and H. Haddadi, "Deep learning in mobile and wireless networking: A survey," IEEE Communications Surveys & Tutorials, vol. 21, no. 3, pp. 2224-2287, 2019.
[21] M. A. Alsheikh, S. Lin, D. Niyato, and H.-P. Tan, "Machine learning in wireless sensor networks: Algorithms, strategies, and applications," IEEE Communications Surveys & Tutorials, vol. 16, no. 4, pp. 1996-2018, 2014.
[22] K. Ovsthus and L. M. Kristensen, "An industrial perspective on wireless sensor networks—A survey of requirements, protocols, and challenges," IEEE communications surveys & tutorials, vol. 16, no. 3, pp. 1391-1412, 2014.
[23] C. Lu et al., "Real-time wireless sensor-actuator networks for industrial cyber-physical systems," Proceedings of the IEEE, vol. 104, no. 5, pp. 1013-1024, 2015.
[24] B. Keswani et al., "Adapting weather conditions based IoT enabled smart irrigation technique in precision agriculture mechanisms," Neural Computing and Applications, vol. 31, no. 1, pp. 277-292, 2019.
[25] A. L. Johann, A. G. de Araújo, H. C. Delalibera, and A. R. Hirakawa, "Soil moisture modeling based on stochastic behavior of forces on a no-till chisel opener," Computers and Electronics in Agriculture, vol. 121, pp. 420-428, 2016.
[26] M. K. Gill, T. Asefa, M. W. Kemblowski, and M. McKee, "Soil moisture prediction using support vector machines 1," JAWRA Journal of the American Water Resources Association, vol. 42, no. 4, pp. 1033-1046, 2006.
[27] L. L. Bello and W. Steiner, "A Perspective on IEEE Time-Sensitive Networking for Industrial Communication and Automation Systems," Proceedings of the IEEE, vol. 107, no. 6, pp. 1094-1120, 2019.
[28] M. Wollschlaeger, T. Sauter, and J. Jasperneite, "The future of industrial communication: Automation networks in the era of the internet of things and industry 4.0," IEEE industrial electronics magazine, vol. 11, no. 1, pp. 17-27, 2017.
[29] S. R. Islam, D. Kwak, M. H. Kabir, M. Hossain, and K.-S. Kwak, "The internet of things for health care: a comprehensive survey," IEEE Access, vol. 3, pp. 678-708, 2015.
[30] M. Verhelst and B. Moons, "Embedded deep neural network processing: Algorithmic and processor techniques bring deep learning to iot and edge devices," IEEE Solid-State Circuits Magazine, vol. 9, no. 4, pp. 55-65, 2017.
[31] Q. Wang et al., "Reducing delay and maximizing lifetime for wireless sensor networks with dynamic traffic patterns," IEEE Access, vol. 7, pp. 70212-70236, 2019.
[32] X. Ge, F. Yang, and Q.-L. Han, "Distributed networked control systems: A brief overview," Information Sciences, vol. 380, pp. 117-131, 2017.
[33] R. V. Kulkarni, A. Forster, and G. K. Venayagamoorthy, "Computational intelligence in wireless sensor networks: A survey," IEEE communications surveys & tutorials, vol. 13, no. 1, pp. 68-96, 2010.
[34] H. Xu, W. Yu, D. Griffith, and N. Golmie, "A survey on industrial Internet of Things: A cyber-physical systems perspective," IEEE Access, vol. 6, pp. 78238-78259, 2018.
[35] Scikit-learn. "Machine Learning in Python." https://scikit-learn.org/stable/ (accessed May 2020).
[36] D. P. K. a. J. L. Ba, "ADAM: A METHOD FOR STOCHASTIC OPTIMIZATION," arXiv:1412.6980v9, 2017.
[Online]. Available: https://arxiv.org/pdf/1412.6980.pdf.
[37] F. Javed, M. K. Afzal, M. Sharif, and B.-S. Kim, "Internet of things (IoT) operating Systems support, networking technologies, applications, and challenges: A comparative review," IEEE Communications Surveys & Tutorials, vol. 20, no. 3, pp. 2062-2100, 2018.
[38] F. Kaup, P. Gottschling, and D. Hausheer, "PowerPi: Measuring and modeling the power consumption of the Raspberry Pi," in 39th Annual IEEE Conference on Local Computer Networks, 2014: IEEE, pp. 236-243.
[39] F. Kaup, S. Hacker, E. Mentzendorff, C. Meurisch, and D. Hausheer, "Energy models for NFV and service provisioning on fog nodes," in NOMS 2018-2018 IEEE/IFIP Network Operations and Management Symposium, 2018: IEEE, pp. 1-7.
[40] Advanticsys. "802.15.4 Mote Modules." https://www.advanticsys.com/shop/802154-mote-modules-c-7_3.html (accessed 2020).