Automatic Music Score Recognition System Using Digital Image Processing

Authors: Yuan-Hsiang Chang, Zhong-Xian Peng, Li-Der Jeng

Abstract:

Music has always been an integral part of human’s daily lives. But, for the most people, reading musical score and turning it into melody is not easy. This study aims to develop an Automatic music score recognition system using digital image processing, which can be used to read and analyze musical score images automatically. The technical approaches included: (1) staff region segmentation; (2) image preprocessing; (3) note recognition; and (4) accidental and rest recognition. Digital image processing techniques (e.g., horizontal /vertical projections, connected component labeling, morphological processing, template matching, etc.) were applied according to musical notes, accidents, and rests in staff notations. Preliminary results showed that our system could achieve detection and recognition rates of 96.3% and 91.7%, respectively. In conclusion, we presented an effective automated musical score recognition system that could be integrated in a system with a media player to play music/songs given input images of musical score. Ultimately, this system could also be incorporated in applications for mobile devices as a learning tool, such that a music player could learn to play music/songs.

Keywords: Connected component labeling, image processing, morphological processing, optical musical recognition.

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1127446

References:

[1] N. Otsu, “A Threshold Selection Method form Gray-Level Histograms,” IEEE Transactions on Systems, pp. 62-66, 1979.
[2] R.G. Casey and E. Lecolinet, “A Survey of Methods and Strategies in Character Segmentation,” IEEE Transactions on Pattern Analysis and Machine Intelligence, pp. 690-706, 1996.
[3] Cheng-Lin Liu, M. Koga and H. Fujisawa, “Lexicon-Driven Segmentation and Recognition of Handwritten Character Strings for Japanese Address Reading,” IEEE Transactions on Pattern Analysis and Machine Intelligence, pp. 1425-1437, 2002.
[4] M. Sotoodeh and F. Tajeripour, “Staff Detection and Removal Using Derivation and Connected Component Analysis,” IEEE 16th CSI International Symposium on Artificial Intelligence and Signal Processing (AISP), pp. 54-57, 2012.
[5] Chen Genfang, Zhang Liyin, Zhang Wenjun and Wang Qiuqiu, “Detecting the Staff-lines of Musical Score with Hough Transform and Mathematical Morphology,” IEEE International Conference on Multimedia Technology (ICMT) , pp. 1-4, 2010.
[6] A. Dutta, U. Pal, A. Fornes and J. Llados, “An Efficient Staff Removal Approach from Printed Musical Documents,” IEEE International Conference on Pattern Recognition (ICPR), pp.1965-1968, 2010.
[7] JaeMyeong Yoo, GiHong Kim and Gueesang Lee, “Mask Matching for Low Resolution Musical Note Recognition,” IEEE International Symposium on Signal Processing and Information Technology, pp. 223-226, 2008.
[8] F.Toyama, K. Shioji and J. Miyamichi, “Symbol Recognition of Printed Piano Scores with Touching Symbols,” Pattern Recognition, ICPR 18th International Conference, pp. 480-483, 2006.
[9] F.Rossant and I. Bloch, “Optical Music Recognition Based on a Fuzzy Modeling of Symbol Classes and Music Writing Rules,” Pattern Recognition Letters, vol.23, pp. 1129-1141, 2002.
[10] K.T. Reed and J.R.Parker, “Automatic Computer Recognition of Printed Music,” in Proceedings of the ICPR, pp.803-807, 1996.