Real Time Detection, Tracking and Recognition of Medication Intake

Authors: H. H. Huynh, J. Meunier, J.Sequeira, M.Daniel

Abstract:

In this paper, the detection and tracking of face, mouth, hands and medication bottles in the context of medication intake monitoring with a camera is presented. This is aimed at recognizing medication intake for elderly in their home setting to avoid an inappropriate use. Background subtraction is used to isolate moving objects, and then, skin and bottle segmentations are done in the RGB normalized color space. We use a minimum displacement distance criterion to track skin color regions and the R/G ratio to detect the mouth. The color-labeled medication bottles are simply tracked based on the color space distance to their mean color vector. For the recognition of medication intake, we propose a three-level hierarchal approach, which uses activity-patterns to recognize the normal medication intake activity. The proposed method was tested with three persons, with different medication intake scenarios, and gave an overall precision of over 98%.

Keywords: Activity recognition, background subtraction, tracking, medication intake, video surveillance

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1086255

References:

[1] D. Batz, M. Batz, N. d. V. Lobo and M. Shah, "A computer vision system for monitoring medication intake," The 2nd Canadian Conference on Computer and Robot Vision, pp. 362-369, 2005.
[2] M. Valin, J. Meunier, A. St-Arnaud and J. Rousseau, "Video surveillance of medication intake," IEEE Engineering in Medicine and Biology Society, vol. 1, pp. 6396-6399, 2006.
[3] S. Ammouri and G.-A. Bilodeau, "Face and hands detection and tracking applied to the monitoring of medication intake," Canadian Conference on Computer and Robot Vision, pp. 147-154, 2008.
[4] S. Birchfield, "Elliptical head tracking using intensity gradients and color histograms," IEEE Conference on Computer Vision and Pattern Recognition, pp. 232-237, 1998.
[5] V. Vezhnevets, V. Sazonov and A. Andreeva, "A survey on pixel-based skin color detection techniques," GRAPHICON-03, pp. 85-92, 2003.
[6] J. Yang, W. Lu and A. Waibel, "Skin-color modeling and adaptation," ACCV'98, pp. 687-694, 1998.
[7] E. Osuna, R. Freund and F. Girosi, "Training support vector machines: an application to face detection," IEEE Conf. Computer Vision and Pattern Recognition, pp. 130-136, 1997.
[8] P. Viola and M. Jones, "Robust real-time object detection," Int. J. Computer Vision, vol. 1, pp. 511-518, 2001.
[9] H. A. Rowley, S. Baluja and T. Kanade, "Neural network-based face detection," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 20, pp. 23-38, 1998.
[10] M. Castrillon-Santana, O. Déeniz-Suarez, L. Anton-Canalis and J.Lorenzo-Navarro, "Face and facial feature detection evaluation: performance evaluation of public domain Haar detectors for face and facial," VISAPP-08, 2008.
[11] H.-H. Huynh, J. Meunier, J. Sequeira and M. Daniel, "Detection and tracking of interest regions for the surveillance of medication intake," GRETSI'09, 2009.
[12] N. Ng Kuang Chern, P. A. Neow and M. H. A. Jr, "Practical issues in pixel-based autofocusing for machine vision," IEEE Int. Conf. Robotics and Automation, vol. 3, pp. 2791-2796, 2001.
[13] R.-L. Hsu, M. Abdel-Mottaleb and A. K.Jain, "Face detection in color images," IEEE Trans Pattern Analysis and Machine Intelligence, vol. 24, pp. 696-706, 2002.
[14] J. A. Nasiri, "A PSO tuning approach for lip detection on color images," 2nd UKSIM European Symposium on Computer Modeling and Simulation EMS'08, pp. 278-282, 2008.
[15] B.D. Zarit, B. J. Super, F. K. H. Quek, "Comparison of five color models in skin pixel classification" Proceedings of the International Workshop on Recognition, Analysis, and Tracking of Faces and Gestures in Real-Time Systems, pp. 58-63, 1999.
[16] www.prosilica.com
[17] C. Nugent et al., "Can technology improve compliance to medication?", Int. Conference on Smart Homes and Health Telematic, Sherbrooke, QC, Canada, pp. 65-72, 2005.
[18] M. K. Hu, "Visual Pattern Recognition by Moment Invariants", IRE Trans. Info. Theory, vol. 8, pp. 179-187, 1962.