Fitness Action Recognition Based on MediaPipe

Authors: Zixuan Xu, Yichun Lou, Yang Song, Zihuai Lin

Abstract:

MediaPipe is an open-source machine learning computer vision framework that can be ported into a multi-platform environment, which makes it easier to use it to recognize human activity. Based on this framework, many human recognition systems have been created, but the fundamental issue is the recognition of human behavior and posture. In this paper, two methods are proposed to recognize human gestures based on MediaPipe, the first one uses the Adaptive Boosting algorithm to recognize a series of fitness gestures, and the second one uses the Fast Dynamic Time Warping algorithm to recognize 413 continuous fitness actions. These two methods are also applicable to any human posture movement recognition.

Keywords: Computer Vision, MediaPipe, Adaptive Boosting, Fast Dynamic Time Warping.

References:

[1] C. Lugaresi et al., “MediaPipe: A Framework for Building Perception Pipelines.” arXiv, Jun. 14, 2019. doi: 10.48550/arXiv.1906.08172.
[2] “Home,” mediapipe. https://google.github.io/mediapipe/ (accessed Oct. 27, 2022).
[3] “Pose,” mediapipe. https://google.github.io/mediapipe/solutions/pose.html (accessed Oct. 27, 2022).
[4] Y. Freund and R. E. Schapire, ”A short introduction to boosting”, J. Jpn. Soc. Artif. Intell., vol. 14, no. 5, pp. 771-780, Sep. 1999.
[5] R. E. Schapire, Y. Freund, P. Bartlett and W. S. Lee, ”Boosting the margin: A new explanation for the effectiveness of voting methods”, Ann. Statist., vol. 26, no. 5, pp. 1651-1686, 1998.
[6] S. Wu and H. Nagahashi, ”Parameterized AdaBoost: Introducing a Parameter to Speed Up the Training of Real AdaBoost,” in IEEE Signal Processing Letters, vol. 21, no. 6, pp. 687-691, June 2014, doi: 10.1109/LSP.2014.2313570.
[7] Y. Freund and R. E. Schapire, “A Decision-Theoretic Generalization of On-Line Learning and an Application to Boosting,” Journal of Computer and System Sciences, vol. 55, no. 1, pp. 119–139, 1997, doi: 10.1006/jcss.1997.1504.
[8] R. E. Schapire and Y. Singer, “Improved boosting algorithms using confidence-rated predictions,” Proceedings of the eleventh annual conference on Computational learning theory - COLT’ 98, 1998.
[9] S. Salvador and P. Chan, “FastDTW: Toward Accurate Dynamic Time Warping in Linear Time and Space,” p. 11.
[10] ”Hi Fitness Action Library: A complete collection of fitness actions, detailed action diagrams and action video teaching!” https://www.hiyd.com/dongzuo/ (accessed Oct. 27, 2022).