Stereotypical Motor Movement Recognition Using Microsoft Kinect with Artificial Neural Network

Authors: M. Jazouli, S. Elhoufi, A. Majda, A. Zarghili, R. Aalouane

Abstract:

Autism spectrum disorder is a complex developmental disability. It is defined by a certain set of behaviors. Persons with Autism Spectrum Disorders (ASD) frequently engage in stereotyped and repetitive motor movements. The objective of this article is to propose a method to automatically detect this unusual behavior. Our study provides a clinical tool which facilitates for doctors the diagnosis of ASD. We focus on automatic identification of five repetitive gestures among autistic children in real time: body rocking, hand flapping, fingers flapping, hand on the face and hands behind back. In this paper, we present a gesture recognition system for children with autism, which consists of three modules: model-based movement tracking, feature extraction, and gesture recognition using artificial neural network (ANN). The first one uses the Microsoft Kinect sensor, the second one chooses points of interest from the 3D skeleton to characterize the gestures, and the last one proposes a neural connectionist model to perform the supervised classification of data. The experimental results show that our system can achieve above 93.3% recognition rate.

Keywords: ASD, stereotypical motor movements, repetitive gesture, kinect, artificial neural network.

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1125077

References:

[1] P. A. Filipek, P. J. Accardo, G. T.Baranek, E. H. Cook, G.Dawson,B. Gordon, et al., “The screening and diagnosis of autistic spectrum disorders”, Journal of Autism and Developmental Disorders, 29 (6), 439–484, 1999.
[2] B. NORIS, “Machine Vision-Based Analysis of Gaze and Visual Context: an Application to Visual Behavior of Children with Autism Spectrum Disorders”,Lausanne, écolepolytechnique fédérale,2011.
[3] J. Stark, “A neural network to compute the Hutchinson metric in fractal image processing”, IEEE Transactions on Neural Networks, 2 (1), 156–158,1991.
[4] J. Jiang, P. Trundle, J. Ren, ”Medical image analysis with artificial neural networks.” Computerized Medical Imaging and Graphics, 34 (8), 617–631, 2010.
[5] K. Saravanan, S. Sasithra, "Review on classification based on artificial neural networks", International Journal of Ambient Systems and Applications (IJASA) Vol.2, No.4, December 2014.
[6] J. A. Munoz-Rodriıguez, A. Asundi, R. Rodriıguez-Vera, "Shape detection of moving objects based on a neural network of a light line", Optics Communications, 221, 73–86, 2003.
[7] F. Albinali, M. S. Goodwin, and S. S. Intille, “Recognizing Stereotypical Motor Movements in the Laboratory and Classroom: A Case Study with Children on the Autism Spectrum,” In: Proceedings of the 11th international conference on Ubiquitous computing, pp. 71–80, 2009.
[8] M. S. Goodwin, S. S. Intille, F. Albinali, and W. F. Velicer, “Automated detection of stereotypical motor movements.”, Journal of autism and developmental disorders, vol. 41, no. 6, pp. 770–82, 2011.
[9] F. Albinali, M. S. Goodwin, and S. S. Intille, “Detecting stereotypical motor movementsin the classroom using accelerometry and pattern recognition algorithms.”, Pervasiveand Mobile Computing, 8(1), 103–114, 2012.
[10] N. Gonçalves, J. L. Rodrigues, S. Costa, and F. Soares, ”Automatic detection of stereotyped hand flapping movements: two different approaches.”, The 21st IEEE International Symposium on Robot and Human Interactive Communication, 2012.
[11] C.H. Min, A.H. Tewfik,“Automatic characterization and detection of behavioralpatterns using linear predictive coding of accelerometer sensor data.”, In: Engineering in Medicine and Biology Society (EMBC), pp. 220–223, 2010.
[12] J. L. Rodrigues, N. Gonc, S. Costa, and F. Soares, ”Stereotyped movement recognition in children with ASD.”,Sensors and Actuators A: Physical, vol. 202, pp. 162–169, 2013.
[13] T. Westeyn, K. Vadas, X. Bian, T. Starner, and G. D. Abowd, “Recognizing Mimicked Autistic Self–Stimulatory Behaviors Using HMMs.”, In: Proceedings. Ninth IEEE International Symposium on pp. 164–167, 2005.
[14] H. Sakoe and S. Chiba,”Dynamic programming algorithm optimization for spoken word recognition”, IEEE Trans. Acoustics, Speech, Signal Processing, 26(1), 43–49, 1978.
[15] C. Myers, Lawrence R. Rabiner and Aaron E. Rosenberg, “Performance Tradeoffs in Dynamic Time Warping Algorithms for Isolated Word Recognition”, IEEE Transactions on Acoustics, Speech and Signal Processing, vol. 28,pp: 623 – 635, 1980.
[16] E. Kolman, M. Margaliot, “Are artificial neural networks white boxes?”, IEEE Transactions on Neural Networks, 16 (4), 844–852, 2005.
[17] S. Haykin, ”Neural Networks and Learning Machines”, 3rd ed. Prentice-Hall, Ontario, 2008.
[18] R. LEPAGE, B. SOLAIMAN, “Les réseaux de neurones artificiels et leurs applications en imagerie et en vision par ordinateur”, Coop ÉTS, 2003.
[19] D. Rebya, S. Lek , I.Dimopoulos, J. Joachim, J. Lauga, S. Aulagnier, ”Artificial neural networks as a classification method in the behavioural sciences”, Behavioural Processes, 40, 35–43, 1997.