Portable Virtual Piano Design

Authors: Yu-Xiang Zhao, Chien-Hsing Chou, Mu-Chun Su, Yi-Zeng Hsieh

Abstract:

The purpose of this study is to design a portable virtual piano. By utilizing optical fiber gloves and the virtual piano software designed by this study, the user can play the piano anywhere at any time. This virtual piano consists of three major parts: finger tapping identification, hand movement and positioning identification, and MIDI software sound effect simulation. To play the virtual piano, the user wears optical fiber gloves and simulates piano key tapping motions. The finger bending information detected by the optical fiber gloves can tell when piano key tapping motions are made. Images captured by a video camera are analyzed, hand locations and moving directions are positioned, and the corresponding scales are found. The system integrates finger tapping identification with information about hand placement in relation to corresponding piano key positions, and generates MIDI piano sound effects based on this data. This experiment shows that the proposed method achieves an accuracy rate of 95% for determining when a piano key is tapped.

Keywords: virtual piano, portable, identification, optical fibergloves.

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1077090

References:

[1] D. J. Sturman and D. Zeltzer, "A Survey of Glove- based Input", IEEE Computer Graphics and Applications, vol.14, no.1 pp. 30-39, 1994.
[2] G. C. Sun, Self-balancing Control and Manipulation of a Glove Puppet Robot on a Two-wheel Mobile Platform, Master's thesis, Institute of Electrical Control Engineering, National Chiao Tung University, 2009.
[3] J. S. Hu, J. J. Wang, and G. Q. Sun, "The Glove Puppet Robot: X-puppet," IEEE/RSJ 2008 International Conference on Intelligent Robots and Systems (IROS -08), Nice, France, 2008.
[4] M. C. Su, C. H. Hung, C. K. Yang, H. T. Yang, Z. J. Huang, Y. X. Zhao, and D. Y. Huang, 2005, "Design of Optical Fiber Glove and Its Application," in 2005 13th National Conference on Fuzzy Theory and Applications, Kaohsing, Taiwan.
[5] J. M. Hollerbach, "Anthropomorphic Robot and Human Interactions", in Proc. of 1st. Intl. Symposium on Humanoid Robots, 1996.
[6] S. S. Fels and G. E. Hinton, "Glove-Talk: a neural network interface between a Data-Glove and a speech synthesizer", IEEE Trans. On Neural Networks, Vol. 1, pp. 2-8, 1993.
[7] M. C. Su, Y. X. Zhao, C. H. Hung, and M. Chunag, "A Fuzzy Rule-Based Approach to Hand Shape Recognition Systems," in The Joint Conference on AI, Fuzzy System, and Grey System, Taiwan, Dec. 2003.
[8] Y. X. Zhao, A Portable Speaking-Aid System for Taiwanese Sign Language, Master's thesis, Department of Electrical Engineering, Tamkang University, 2002.
[9] C. H. Hung, A Trajectory-based Approach to Gesture Recognition, Master's thesis, Department of Computer Science and Information, Engineering, National Central University, 2006.
[10] M. C. Su, Y. X. Zhao, H. Huang, and H. F. Chen, "A Fuzzy Rule-Based Approach to Recognizing 3D Arm Movements," IEEE Trans. on Neural Systems and Rehabilitation Engineering, vol.9, no. 2, pp. 191-201, June 2001.
[11] M. C. Su, C. H. Hung, and Y. X. Zhao, "A SOMART System for Gesture Recognition," WSEAS Transactions on Computers, vol. 5, pp. 2764-2771, 2006.
[12] V. Tartter and K. Knowlton, "Perception of sign language from an array of 27 moving spots," Nature, vol. 2, pp. 676-678, 1981.
[13] J. Weissmann and R. Salomon, "Gesture recognition for virtual reality applications using data gloves and neural networks," Proceeding of the International Joint Conference on Neural Networks, Vol.3, pp.2043-2046, 1999.
[14] G. Burdea, S. Deshpande, B. Liu, N. Langrana, and D. Gomez, "A Virtual Reality-Base System for Hand Diagnosis and Rehabilitation," IEEE Transaction on Rehabilitation Engineering, vol. 6, no. 2, pp. 229-240, 1997.
[15] V. Popescu, G. Burdea, and M. Bouzit, "Virtual Reality Simulation for a Haptic Glove," IEEE Computer Animation Proceedings, Geneva, Switzerland, pp. 195-200, 1999.
[16] V. Popescu, and G. Burdea, "A Virtual-Reality-Base Telerehabilitation System with Force Feedback," IEEE Transactions on Information Technology in Biomedicine, vol. 4, Issue: 1, pp. 45-51, March 2000.
[17] Power Glove, http://en.wikipedia.org/wiki/Power_Glove.
[18] 5DT Data Glove & CyberGlove, http://www.vrlogic.com/html/datagloves.html.
[19] S. Vance, L. Migachyov, and W. Fu, I.Hajjar, "Fingerless glove for Interacting with data processing system", United States Patent, Patten No: 6,304,840, Oct.16, 2001.