Learning Programming for Hearing Impaired Students via an Avatar

Authors: Nihal Esam Abuzinadah, Areej Abbas Malibari, Arwa Abdulaziz Allinjawi, Paul Krause

Abstract:

Deaf and hearing-impaired students face many obstacles throughout their education, especially with learning applied sciences such as computer programming. In addition, there is no clear signs in the Arabic Sign Language that can be used to identify programming logic terminologies such as while, for, case, switch etc. However, hearing disabilities should not be a barrier for studying purpose nowadays, especially with the rapid growth in educational technology. In this paper, we develop an Avatar based system to teach computer programming to deaf and hearing-impaired students using Arabic Signed language with new signs vocabulary that is been developed for computer programming education. The system is tested on a number of high school students and results showed the importance of visualization in increasing the comprehension or understanding of concepts for deaf students through the avatar.

Keywords: Hearing-impaired students, isolation, self-esteem, learning difficulties.

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.3669204

References:

[1] T. van Gent, A. Goedhart, H. Knoors, P. Westenberg and P. Treffers, “Self-concept and Ego Development in Deaf Adolescents: A Comparative Study,” Journal of Deaf Studies and Deaf Education, vol. 17, no. 3, pp. 333-351, 2012.
[2] C. Bisol, C. Valentini, J. Simioni and J. Zanchin, “Deaf Students in Higher Education: Reflections On Inclusion,” 172 Cadernos de Pesquisa, vol. 40, no. 139, 2010.
[3] H. Lang, “Higher Education for Deaf Students: Research Priorities in the New Millennium”, Journal of Deaf Studies and Deaf Education, vol. 7, no. 4, pp. 267-280, 2002.
[4] M. Boulares and M. Jemni, “3D Motion Trajectory Analysis Approach to Improve Sign Language 3D-based Content Recognition’, Procedia Computer Science, vol. 13, pp. 133-143, 2012.
[5] B. Faraj, W. Al-Rajhi, and Y. Elhadj. “Avatar Based Approach for Teaching Arabic Sign Language,” Journal of Communications and Computer Engineering, vol. 2 no. 2, pp. 43-48, 2011.
[6] Washington.edu, “How are the terms deaf, deafened, hard of hearing, and hearing impaired typically used? | DO-IT,” 2015. (Online). Available: http://www.washington.edu/doit/how-are-terms-deaf-deafened-hard-hearing-and-hearing-impaired-typically-used. (Accessed: 18- Oct- 2015).
[7] T. Wareham, G. Clark, and C. Laugesen. Providing learning support for d/Deaf or hearing impaired students undertaking fieldwork and related activities. Cheltenham and Gloucester College of Higher Education, Geography Discipline Network (GDN), 2001.
[8] World Health Organization. “WHO global estimates on prevalence of hearing loss.” Mortality and Burden of Diseases and Prevention of Blindness and Deafness. WHO 2012.
[9] R. Mitchell, “How Many Deaf People Are There in the United States? Estimates From the Survey of Income and Program Participation,” Journal of Deaf Studies and Deaf Education, vol. 11, no. 1, pp. 112-119, 2005.
[10] World Health Organization. ‘WHO | Deafness and hearing loss’, 2015. (Online). Available: http://www.who.int/mediacentre/factsheets/fs300/en/.
[11] K.J. Daghistani, T.S. Jamal, and S.M. Zakzouk, “The management of hearing impaired Saudi children: An epidemiological survey,” Bahrain Medical Bulletin, vol. 24 no. 1, pp. 7-12, 2002.
[12] J. Sarah Hamrick, “Asia, the Middle East, and Oceania - Deaf Statistics - LibGuides at Gallaudet University Library.” Libguides.gallaudet.edu, 2015. (Online). Available: http://libguides.gallaudet.edu/content.php?pid=119476&sid=1061104.
[13] Secretariat General. “Global Survey Report WFD Regional Secretariat for Mexico, Central America and the Caribbean Global Education Pre-Planning Project on the Human Rights of Deaf People Compiled by Mr Colin Allen Project Co-ordinator.”
[14] Hesc.mohe.gov.sa, ‘نظام احصاءات التعليم’, 2015. (Online). Available: https://hesc.mohe.gov.sa/pages/default.aspx.
[15] C. Erting, The deaf way. Washington, D.C.: Gallaudet Univ. Press, 1994. Page 275-276
[16] A. Elsayed and H. Hamdy, “Arabic Sign Language (ArSL) Recognition System Using HMM,” International Journal of Advanced Computer Science and Applications, vol. 2, no. 11, 2011.
[17] G. University, “Fast Facts - Gallaudet University,” Gallaudet.edu, 2015. (Online). Available: https://www.gallaudet.edu/about-gallaudet/fast-facts.html.
[18] Ntid.rit.edu, “RIT - NTID – Overview,” 2015. (Online). Available: http://www.ntid.rit.edu/about.
[19] Deaf-trust.co.uk, “About : Communication Specialist College,” 2015. (Online). Available: http://www.deaf-trust.co.uk/college/about/.
[20] Tsukuba-tech.ac.jp, “National University Corporation Tsukuba University of Technology | ABOUT,” 2015. (Online). Available: http://www.tsukuba-tech.ac.jp/english/about/.
[21] D. Armstrong, Show of hands. Washington, DC: Gallaudet University Press, 2011.
[22] Sandler, Wendy, and Diane Lillo-Martin. “Natural sign languages.” Handbook of linguistics, pp. 533-562, 2001.
[23] Cox, Stephen, et al. “Tessa, a system to aid communication with deaf people.”Proceedings of the fifth international ACM conference on Assistive technologies. ACM, 2002.
[24] A. Elsayed and H. Hamdy, ‘Arabic Sign Language (ArSL) Recognition System Using HMM’, International Journal of Advanced Computer Science and Applications, vol. 2, no. 11, 2011.
[25] E. Blake, W. Tucker and M. Glaser, “Towards Communication and Information Access for Deaf People,” SACJ, vol. 54, 2014.
[26] M. Debevc, Z. Stjepanovič and A. Holzinger, “Development and evaluation of an e-learning course for deaf and hard of hearing based on the advanced Adapted Pedagogical Index method,” Interactive Learning Environments, vol. 22, no. 1, pp. 35-50, 2012.
[27] Cooper, Helen, Brian Holt, and Richard Bowden. “Sign language recognition.” Visual Analysis of Humans. Springer London, 2011. 539-562.
[28] O. Al-Jarrah and A. Halawani, “Recognition of gestures in Arabic sign language using neuro-fuzzy systems,” Artificial Intelligence, vol. 133, no. 1-2, pp. 117-138, 2001.
[29] M.J.O. ELGHOUL, “An avatar based approach for automatic interpretation of text to Sign language.” 9th European Conference for the Advancement of the Assistive Technologies in Europe. 2007.
[30] S.Andrei, L. Osborne, and Z. Smith, “Designing an American Sign Language Avatar for Learning Computer Science Concepts for Deaf or Hard-of-Hearing Students and Deaf Interpreters,” Journal of Educational Multimedia and Hypermedia, vol. 22 no. 3, pp. 229-242, 2013.
[31] J. Joel Theard, “Vcom3D, Inc | Vcommunicator,” Vcom3d.com, 2015. (Online). Available: http://www.vcom3d.com/vcommunicator.php.
[32] Visicast.cmp.uea.ac.uk, ‘eSIGN at UEA’, 2015. (Online). Available: http://www.visicast.cmp.uea.ac.uk/eSIGN/.
[33] ElGhoul, Oussama, and Mohamed Jemni. “WebSign: A system to make and interpret signs using 3D Avatars.” Proceedings of the Second International Workshop on Sign Language Translation and Avatar Technology (SLTAT), Dundee, UK. 2011.
[34] Al-Nafjan, Abeer, and Yousef Al-Ohali. “A Multimedia System for Learning Arabic Sign Language: Tawasoul.” World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education, vol. 2010, no. 1. 2010.
[35] N. Adamo-Villani, E. Carpenter, and L. Arns. “An immersive virtual environment for learning sign language mathematics.” ACM SIGGRAPH 2006 Educators program. ACM, 2006.
[36] R.K. Dean, and Q. Robert. “Application of demand-control theory to sign language interpreting: Implications for stress and interpreter training.” Journal of deaf studies and deaf education,pp. 1-14, 2001.
[37] M. P. Mishra, K.S. Vinay, and R. Tripathi. “ICT as a Tool for Teaching and Learning in Respect of Learner with Disability.” Indira Gandhi National Open University, New Delhi, India, 2014.
[38] C. Blurton. “New directions of ICT-use in education.” Retrieved on 24 (1999): 2012.
[39] J.A. Kulik, Effects of using instructional technology in elementary and secondary schools: What controlled evaluation studies say. Arlington, VA: SRI International, 2003.
[40] A.S. Drigas, D. Kouremenos, S. Kouremenos, J. and Vrettaros, “An e-Learning System for the Deaf people.” Information Technology Based Higher Education and Training, 2005. ITHET 2005. 6th International Conference on. IEEE, 2005.
[41] O. Bauer. “The question of nationalities and social democracy”. U of Minnesota Press; 2000.
[42] M.F Tolba, A Samir, M Abul-Ela. “A proposed graph matching technique for Arabic sign language continuous sentences recognition”. InInformatics and Systems (INFOS), 2012 8th International Conference on 2012 May 14 (pp. MM-14). IEEE.
[43] N.K Ibrahim, M.F Noordin, S. Samsuri, M.S Seman, A.E Ali. “Isnad Al-Hadith Computational Authentication: An Analysis Hierarchically”. InInformation and Communication Technology for The Muslim World (ICT4M), 2016 6th International Conference on 2016 Nov 22 (pp. 344-348). IEEE.
[44] R.M Felder, L.K Silverman. “Learning and teaching styles in engineering education,” Engineering education, vol. 78, no, 7, pp, 674-681, 1988.
[45] M. Marschark. “Psychological development of deaf children”. Oxford University Press On Demand; 1997.
[46] El, A. E. E., M. M. R. El, and S. M. El Atawy. "Intelligent Arabic text to arabic sign language translation for easy deaf communication." International Journal of Computer Applications 92.8 (2014).
[47] M Khan, SS Khan. “Data and information visualization methods, and interactive mechanisms: A survey,” International Journal of Computer Applications, vol. 34, no. 1, pp. 1-4, 2011.
[48] K Verbert, E Duval, J Klerkx, S Govaerts, J.L Santos. “Learning analytics dashboard applications,” American Behavioral Scientist, vol. 57, no. 10, pp. 1500-1509, 2013.
[49] E Cambria, B White. “Jumping NLP curves: A review of natural language processing research”. IEEE Computational intelligence magazine. Vol. 9, no. 2, pp.48-57, 2014.
[50] A Copestake, P Corbett, P Murray-Rust, C.P Rupp, A Siddharthan, S Teufel, B Waldron. “An architecture for language processing for scientific texts”. InProceedings of the UK e-Science All Hands Meeting 2006 2006.
[51] J. Neves, A Translation, J.N Ba. “Subtitling for the Deaf and Hard-of-Hearing”.
[52] A Dyer, M MacSweeney, M Szczerbinski M, Green L, Campbell R. “Predictors of reading delay in deaf adolescents: The relative contributions of rapid automatized naming speed and phonological awareness and decoding,” Journal of Deaf Studies and Deaf Educatio, vol. 8, no. 3, pp. 215-229, 2003.
[53] N Indurkhya, F.J Damerau, editors. “Handbook of natural language processing”. CRC Press; 2010 Feb 22.
[54] AE, El, MM El, SM El Atawy. “Intelligent Arabic text to Arabic sign language translation for easy deaf communication,” International Journal of Computer Applications, vol. 92, no. 9, 2014.
[55] K.A. Weaver, and T. Starner. We need to communicate!: helping hearing parents of deaf children learn American Sign Language. InThe proceedings of the 13th international ACM SIGACCESS Conference on Computers and Accessibility 2011 Oct 24 (pp. 91-98). ACM.