An Embedded Vision Solution for Joint Localization: Application in Soft Robotic Rehabilitation Gloves

Authors: Narges Ghobadi, Witold Kinsner, Tony Szturm, Nariman Sepehri

Abstract:

Accurate joint localization is essential for effective hand rehabilitation using soft robotic gloves. This paper presents a real-time joint localization system that employs a lightweight object detection model (YOLOv8n), deployed on a micro-computer integrated with an AI accelerator. Visual tags embedded on the glove enable precise detection and tracking of finger movements. The model, trained to recognize these tags with 99% precision, achieves low-latency inference of approximately 3 ms per frame and operates at an average frame rate of 60 frames per second (fps). This corresponds to a Nyquist frequency of 30 Hz, ensuring accurate capture of human hand movements without aliasing. The system's compact and energy-efficient design supports portability and ease of use for patients, while real-time data processing enables responsive feedback during rehabilitation exercises. Performance evaluations confirm the system’s effectiveness under the computational constraints of embedded platforms.

Keywords: Embedded system, vision system, marker-based detection, rehabilitation, joint localization.

References:

[1] N. Ghobadi, N. Sepehri, W. Kinsner, and T. Szturm, “Beyond Human Touch: Integrating Soft Robotics with Environmental Interaction for Advanced Applications,” in Actuators, MDPI, 2024, p. 507.
[2] T. Hellsten, J. Karlsson, M. Shamsuzzaman, and G. Pulkkis, “The potential of computer vision-based marker-less human motion analysis for rehabilitation,” Rehabilitation Process and Outcome, vol. 10, p. 11795727211022330, 2021.
[3] M. H. Khan, M. Zöller, M. S. Farid, and M. Grzegorzek, “Marker-based movement analysis of human body parts in therapeutic procedure,” Sensors, vol. 20, no. 11, p. 3312, 2020.
[4] A. Kolahi, M. Hoviattalab, T. Rezaeian, M. Alizadeh, M. Bostan, and H. Mokhtarzadeh, “Design of a marker-based human motion tracking system,” Biomed Signal Process Control, vol. 2, no. 1, pp. 59–67, 2007.
[5] M. A. Ali, K. Sundaraj, R. B. Ahmad, N. U. Ahamed, and M. A. Islam, “Vision-based motion tracking rehabilitation system for gait disorder,” in 2012 IEEE International Conference on Control System, Computing and Engineering, IEEE, 2012, pp. 371–375.
[6] L. De Vito, O. Postolache, and S. Rapuano, “Measurements and sensors for motion tracking in motor rehabilitation,” IEEE Instrum Meas Mag, vol. 17, no. 3, pp. 30–38, 2014.
[7] V. Sholukha, B. Bonnechere, P. Salvia, F. Moiseev, M. Rooze, and S. V. S. Jan, “Model-based approach for human kinematics reconstruction from markerless and marker-based motion analysis systems,” J Biomech, vol. 46, no. 14, pp. 2363–2371, 2013.
[8] S. L. Colyer, M. Evans, D. P. Cosker, and A. I. T. Salo, “A review of the evolution of vision-based motion analysis and the integration of advanced computer vision methods towards developing a markerless system,” Sports medicine-open, vol. 4, pp. 1–15, 2018.
[9] N. Ghobadi, M. Khajoee, W. Kinsner, T. Szturm, and N. Sepehri, “Real-time Interfacing of a Pneumatically-actuated Finger-thumb Rehabilitation Device”.
[10] M. Sierotowicz et al., “EMG-driven machine learning control of a soft glove for grasping assistance and rehabilitation,” IEEE Robot Autom Lett, vol. 7, no. 2, pp. 1566–1573, 2022.
[11] G. B. Prange-Lasonder, B. Radder, A. I. R. Kottink, A. Melendez-Calderon, J. H. Buurke, and J. S. Rietman, “Applying a soft-robotic glove as assistive device and training tool with games to support hand function after stroke: Preliminary results on feasibility and potential clinical impact,” in 2017 International Conference on Rehabilitation Robotics (ICORR), IEEE, 2017, pp. 1401–1406.
[12] S. Ghate, L. Yu, K. Du, C. T. Lim, and J. C. Yeo, “Sensorized fabric glove as game controller for rehabilitation,” in 2020 IEEE SENSORS, IEEE, 2020, pp. 1–4.
[13] J. Chen, Y. Gao, and S. Li, “Real-time Apriltag inertial fusion localization for large indoor navigation,” in 2020 Chinese Automation Congress (CAC), IEEE, 2020, pp. 6912–6916.
[14] Z. Wu, Y. Zhang, X. Wang, H. Li, Y. Sun, and G. Wang, “Algorithm for detecting surface defects in wind turbines based on a lightweight YOLO model,” Sci Rep, vol. 14, no. 1, p. 24558, 2024.
[15] A. Kojima, “Development of a Miniature Self-Driving Car Using SoC and NPU,” in 2025 IEEE International Conference on Consumer Electronics (ICCE), IEEE, 2025, pp. 1–2.
[16] R. Varghese and M. Sambath, “Yolov8: A novel object detection algorithm with enhanced performance and robustness,” in 2024 International Conference on Advances in Data Engineering and Intelligent Computing Systems (ADICS), IEEE, 2024, pp. 1–6.
[17] N. Sun, G. Li, and L. Cheng, “Design and Validation of a Self-Aligning Index Finger Exoskeleton for Post-Stroke Rehabilitation,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 29, pp. 1513–1523, 2021, doi: 10.1109/TNSRE.2021.3097888.
[18] E. O. Brigham and R. E. Morrow, “The fast Fourier transform,” IEEE Spectr, vol. 4, no. 12, pp. 63–70, 1967.
[19] J. Groß et al., “The neural basis of intermittent motor control in humans,” Proceedings of the National Academy of Sciences, vol. 99, no. 4, pp. 2299–2302, 2002.
[20] G. P. van Galen and M. van Huygevoort, “Error, stress and the role of neuromotor noise in space oriented behaviour,” Biol Psychol, vol. 51, no. 2–3, pp. 151–171, 2000.
[21] A. M. Halliday and J. W. T. Redfearn, “An analysis of the frequencies of finger tremor in healthy subjects,” J Physiol, vol. 134, no. 3, p. 600, 1956.