Preliminary Study of Hand Gesture Classification in Upper-Limb Prosthetics Using Machine Learning with Electromyographic Signals

Authors: Linghui “Jeff” Meng, James Atlas, Deborah Munro

Abstract:

There is an increasing demand for prosthetics capable of mimicking natural limb movements and hand gestures, but precise movement control of prosthetics using only electrode signals continues to be challenging. This study considers the implementation of machine learning as a means of improving accuracy and presents an initial investigation into hand gesture recognition using models based on electromyographic (EMG) signals. EMG signals, which capture muscle activity, are used as inputs to machine learning algorithms to improve prosthetic control accuracy, functionality, and adaptivity. Using logistic regression, a machine learning classifier, this study evaluates the accuracy of classifying two hand gestures from the publicly available Ninapro dataset using two-time series feature extraction algorithms: Time Series Feature Extraction (TSFE) and Convolutional Neural Networks (CNNs). Trials were conducted using varying numbers of EMG channels from one to eight to determine the impact of channel quantity on classification accuracy. The results suggest that although both algorithms can successfully distinguish between hand gesture EMG signals, CNNs outperform TSFE in extracting useful information for both accuracy and computational efficiency. In addition, although more channels of EMG signals provide more useful information, they also require more complex and computationally intensive feature extractors and consequently do not perform as well as lower numbers of channels. The findings also underscore the potential of machine learning techniques in developing more effective and adaptive prosthetic control systems.

Keywords: EMG, machine learning, prosthetic control, electromyographic prosthetics, hand gesture classification, CNN, computational neural networks, TSFE, time series feature extraction, channel count, logistic regression, ninapro, classifiers.

References:

[1] E. A. Biddiss and T. T. Chau, “Upper limb prosthesis use and abandonment: A survey of the last 25 years,” Prosthet. Orthot. Int., vol. 31, no. 3, pp. 236–257, Sep. 2007, doi: 10.1080/03093640600994581.
[2] E. Biddiss and T. Chau, “Upper-Limb Prosthetics: Critical Factors in Device Abandonment,” Am. J. Phys. Med. Rehabil., vol. 86, no. 12, pp. 977–987, Dec. 2007, doi: 10.1097/PHM.0b013e3181587f6c.
[3] E. Biddiss, D. Beaton, and T. Chau, “Consumer design priorities for upper limb prosthetics,” Disabil. Rehabil. Assist. Technol., vol. 2, no. 6, pp. 346–357, Jan. 2007, doi: 10.1080/17483100701714733.
[4] R. Merletti and D. Farina, Eds., Surface electromyography: physiology, engineering and applications. in IEEE Press series in biomedical engineering. Piscataway, NJ: Hoboken, New Jersey: IEEE Press ; Wiley, 2016.
[5] G. J. Tortora and B. Derrickson, Principles of anatomy & physiology, 14th edition. Danvers, MA: Wiley, 2014.
[6] P. Capsi-Morales et al., “Functional assessment of current upper limb prostheses: An integrated clinical and technological perspective,” PLOS ONE, vol. 18, no. 8, p. e0289978, Aug. 2023, doi: 10.1371/journal.pone.0289978.
[7] M. R. Mohebbian et al., “A Comprehensive Review of Myoelectric Prosthesis Control”.
[8] S. Bahrami Moqadam, S. M. Elahi, A. Mo, and W. Zhang, “Hybrid control combined with a voluntary biosignal to control a prosthetic hand,” Robot. Biomim., vol. 5, no. 1, p. 4, Dec. 2018, doi: 10.1186/s40638-018-0087-5.
[9] N. Thomas, G. Ung, C. McGarvey, and J. D. Brown, “Comparison of vibrotactile and joint-torque feedback in a myoelectric upper-limb prosthesis,” J. NeuroEngineering Rehabil., vol. 16, no. 1, p. 70, Dec. 2019, doi: 10.1186/s12984-019-0545-5.
[10] Á. L. Valdivieso Caraguay, J. P. Vásconez, L. I. Barona López, and M. E. Benalcázar, “Recognition of Hand Gestures Based on EMG Signals with Deep and Double-Deep Q-Networks,” Sensors, vol. 23, no. 8, p. 3905, Apr. 2023, doi: 10.3390/s23083905.
[11] A. M. Moslhi, H. H. Aly, and M. ElMessiery, “The Impact of Feature Extraction on Classification Accuracy Examined by Employing a Signal Transformer to Classify Hand Gestures Using Surface Electromyography Signals,” Sensors, vol. 24, no. 4, p. 1259, Feb. 2024, doi: 10.3390/s24041259.
[12] A. Jung, Machine Learning: The Basics. in Machine Learning: Foundations, Methodologies, and Applications. Singapore: Springer Nature Singapore, 2022. doi: 10.1007/978-981-16-8193-6.
[13] C. M. Bishop, Pattern recognition and machine learning. in Information science and statistics. New York: Springer, 2006.
[14] A. Jaramillo-Yánez, M. E. Benalcázar, and E. Mena-Maldonado, “Real-Time Hand Gesture Recognition Using Surface Electromyography and Machine Learning: A Systematic Literature Review,” Sensors, vol. 20, no. 9, p. 2467, Apr. 2020, doi: 10.3390/s20092467.
[15] S. Bhardwaj, A. A. Khan, and M. Muzammil, “Electromyography in Physical Rehabilitation: A Review,” 2016.
[16] M. Barandas et al., “TSFEL: Time Series Feature Extraction Library,” SoftwareX, vol. 11, p. 100456, Jan. 2020, doi: 10.1016/j.softx.2020.100456.
[17] J. Brownlee, Logistic Regression for Machine Learning. 2019. Online. Available: https://machinelearningmastery.com/logistic-regression-for-machine-learning/
[18] F. Pampel, Logistic Regression. 2455 Teller Road, Thousand Oaks California 91320 United States of America: SAGE Publications, Inc., 2000. doi: 10.4135/9781412984805.
[19] M. Jogin, Mohana, M. S. Madhulika, G. D. Divya, R. K. Meghana, and S. Apoorva, “Feature Extraction using Convolution Neural Networks (CNN) and Deep Learning,” in 2018 3rd IEEE International Conference on Recent Trends in Electronics, Information & Communication Technology (RTEICT), Bangalore, India: IEEE, May 2018, pp. 2319–2323. doi: 10.1109/RTEICT42901.2018.9012507.
[20] Z. Qin, Z. He, Y. Li, S. Saetia, and Y. Koike, “A CW-CNN regression model-based real-time system for virtual hand control,” Front. Neurorobotics, vol. 16, p. 1072365, Dec. 2022, doi: 10.3389/fnbot.2022.1072365.
[21] U. Côté-Allard et al., “Deep Learning for Electromyographic Hand Gesture Signal Classification Using Transfer Learning,” Jan. 25, 2019, arXiv: arXiv:1801.07756. Accessed: Jul. 28, 2022. Online. Available: http://arxiv.org/abs/1801.07756
[22] C. C. Aggarwal, Neural Networks and Deep Learning: A Textbook. Cham: Springer International Publishing, 2018. doi: 10.1007/978-3-319-94463-0.
[23] Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” Nature, vol. 521, no. 7553, pp. 436–444, May 2015, doi: 10.1038/nature14539.
[24] F. Alvi, “PyTorch vs TensorFlow in 2024: A Comparative Guide of AI Frameworks.” Online. Available: https://opencv.org/blog/pytorch-vs-tensorflow/
[25] M. R. Azghadi et al., “Hardware Implementation of Deep Network Accelerators Towards Healthcare and Biomedical Applications,” IEEE Trans. Biomed. Circuits Syst., vol. 14, no. 6, pp. 1138–1159, Dec. 2020, doi: 10.1109/TBCAS.2020.3036081.
[26] A. Manfredo, “Ninapro Database.” 2017. Online. Available: http://ninaweb.hevs.ch/node/7
[27] M. Atzori et al., “Building the Ninapro database: A resource for the biorobotics community,” in 2012 4th IEEE RAS & EMBS International Conference on Biomedical Robotics and Biomechatronics (BioRob), Rome, Italy: IEEE, Jun. 2012, pp. 1258–1265. doi: 10.1109/BioRob.2012.6290287.
[28] M. Atzori et al., “Electromyography data for non-invasive naturally-controlled robotic hand prostheses,” Sci. Data, vol. 1, no. 1, p. 140053, Dec. 2014, doi: 10.1038/sdata.2014.53.
[29] A. K. Jean, M. Diarra, B. A. Bakary, G. Pierre, and A. K. Jérôme, “Application based on Hybrid CNN-SVM and PCA-SVM Approaches for Classification of Cocoa Beans,” Int. J. Adv. Comput. Sci. Appl., vol. 13, no. 9, 2022, doi: 10.14569/IJACSA.2022.0130927.
[30] F. Pedregosa et al., “Scikit-learn: Machine Learning in Python,” Mach. Learn. PYTHON, 2011.
[31] Y. Sasaki, “The truth of the F-measure,” 2007.
[32] D. Powers, “Evaluation: From Precision, Recall and F-Factor to ROC, Informedness, Markedness & Correlation,” 2011.