On a Theoretical Framework for Language Learning Apps Evaluation

Authors: Juan Manuel Real-Espinosa

Abstract:

This paper addresses the first step to evaluate language learning apps: what theoretical framework to adopt when designing the app evaluation framework. The answer is not just one, since there are several options that could be proposed. However, the question to be clarified is to what extent the learning design of apps is based on a specific learning approach, or on the contrary, on a fusion of elements from several theoretical proposals and paradigms, such as m-learning, Mobile Assisted Language Learning and a number of theories about language acquisition. The present study suggests that the reality is closer to the second assumption. This implies that the theoretical framework against which the learning design of the apps should be evaluated, must also be a hybrid theoretical framework, which integrates evaluation criteria from the different theories involved in language learning through mobile applications.

Keywords: Action-oriented approach, apps evaluation, mobile-assisted language learning, post-method pedagogy.

References:

[1] Kim, H., and Kwon, Y. (2012) 'Exploring smartphone applications for effective Mobile-Assisted Language Learning', Multimedia-Assisted Language Learning, 16(1), pp. 31–57.
[2] Son, J. B. (2016) ‘Selecting and evaluation mobile Apps for language learning’, In Palalas, A. and Ally, M. (Eds.). The international handbook of mobile-assisted language. Beijing: China Central Radio & TV University Press, pp. 161-179.
[3] Castañeda, D.A. and Cho, M.H. (2016) 'Use of a game-like application on a mobile device to improve accuracy in conjugating Spanish verbs', Computer Assisted Language Learning, 29(7), pp. 1195–1204.
[4] Kétyi, A. (2013) 'Using Smart Phones in Language Learning – A Pilot Study to Turn CALL into MALL', in Bradley, L. and Thouësny, S. (Eds.), 20 Years of EUROCALL: Learning from the Past, Looking to the Future. Proceedings of the 2013 EUROCALL Conference, Évora, Portugal, pp. 129-134.
[5] Kétyi, A. (2015) 'Practical evaluation of a mobile language learning tool in higher education', in Helm, F., Bradley, L., Guarda, M. and Thouësny, S. (Eds.) Critical CALL – Proceedings of the 2015 EUROCALL Conference, pp. 306-311.
[6] Elaish, M.M., Shuib, L., Ghani, N.A., Yadegaridehkordi E., and Alaa, M. (2017) 'M-learning for English Language Acquisition: Taxonomy, Challenges, and Recommendations', IEEE Acces, 5, pp. 19033-19047.
[7] Littlejohn, A. (1998) ‘The analysis of language teaching materials: Inside the Trojan horse’, in Tomlinson, B. (ed.) Materials Development in Language Teaching. Cambridge: Cambridge University Press, pp. 190-216.
[8] Calle-Martínez, C., Rodríguez-Arancón, P., and Arús-Hita, J. (2014) 'A scrutiny of the educational value of EFL m-learning applications', in Proceedings of World Conference on Learning, Teaching and Educational Leadership 2013.
[9] Chen, X.-B. (2013) ‘Tablets for informal language learning: Student usage and attitudes’, Language, Learning and Technology, 17(1), pp. 20-36.
[10] Reinders, H., & Pegrum, M. (2016). Supporting language learning on the move. An evaluative framework for mobile language learning resources. In B. Tomlinson (Ed.), SLA Research and Materials Development for Language Learning (pp. 116-141). New York, NY: Routledge.
[11] Rosell-Aguilar, F. (2017) 'State of the app: A taxonomy and framework for evaluating language learning mobile applications', The CALICO Journal, 34(2), pp. 243–258.
[12] Crompton, H. (2013) 'A Historical Overview of M-learning: Toward Learner-Centered Education', Handbook of M-learning. Florence: Routledge, pp. 3-14.
[13] Quinn, C. (2000) 'mLearning: Mobile, wireless, in-your-pocket learning', LiNE Zine. Available online http://www.linezine.com/2.1/features/cqmmwiyp.htm (Accessed 3rd June 2018).
[14] Traxler, J. (2005) ‘M-learning-it’s here but what is it?’, Interactions 9(1) University of Warwick. Available online https://warwick.ac.uk/services/ldc/resource/interactions/issues/issue25/traxler/ (Accessed 3rd August 2018).
[15] Brown, T. H. (2005) ‘Towards a model for m-learning in Africa’, International Journal of E-Learning, 4(3), pp. 299-315.
[16] Peters, K. (2007) 'm-Learning: Positioning educators for a mobile, connected future', International Journal of Research in Open and Distance Learning, 8(2), pp. 1–17.
[17] Sharples, M. (2005) ‘Learning as conversation: Transforming education in the mobile age’, Proceedings of Seeing, Understanding, Learning in the Mobile Age, pp. 147–152.
[18] Taylor, J., Sharples, M., O’Malley, C., Vavoula, G. and Waycott, J. (2006) ‘Towards a task model for mobile learning: A dialectical approach’, International Journal of Learning Technology, 2, pp. 138–158.
[19] Kakihara, M. and Sørensen, C. (2002) 'Mobility: An Extended Perspective', 35th Hawaii International Conference on System Sciences, Big Island, Hawaii, pp., 1756–1766.
[20] Kukulska-Hulme, A. (2009) 'Will mobile learning change language learning?' ReCALL, 21(2), pp. 157-165.
[21] Sharples, M. and Pea, R. (2014) 'Mobile learning', in Sawyer, R. K. (Ed.) The Cambridge Handbook of the Learning Sciences (2nd edn) New York: Cambridge University Press, pp. 501-521.
[22] Sharples, M., Taylor, J., Vavoula, G.N. (2007) 'A Theory of Learning for the Mobile Age', in Andrews, R., and Haythornthwaite, C. (Eds.) The SAGE Handbook of E-learning Research. London: Sage, pp. 221-47.
[23] Lai, K. W., Khaddaget, F., and Knezek, G. (2013) 'Blending student technology experiences in formal and informal learning', Journal of Computer Assisted Learning, 29, pp. 414–425.
[24] UNESCO (2013) Policy Guidelines for Mobile Learning. Paris: United Nations Educational, Scientific and Cultural Organization. Available online http://unesdoc.unesco.org/images/0021/002196/219641e.pdf (Accessed 28th June 2018).
[25] Klopfer, E., Squire, K. and Jenkins, H. (2002) ‘Environmental detectives: PDAs as a window into a virtual simulated world’, International Workshop on Wireless and Mobile Technologies in Education, Växjö, Sweden.
[26] Chinnery, G. M. (2006) 'Emerging technologies going to the MALL: Mobile assisted language learning', Language Learning & Technology, 10(1), pp. 9-16.
[27] Laurillard, D. (2007) 'Pedagogical forms for m-learning: Framing research questions', in Pachler, N. (Ed.) M-learning: Towards a research agenda. London: WLE Centre, the Institute of Education, University of London, pp. 153–175.
[28] Viberg, O., & Grönlund, Å. (2012). 'Mobile assisted language learning: a literature review', in 11th international conference on mobile and contextualised learning, pp. 9–16.
[29] Trifonova, A., Knapp, J., Ronchetti, M. and Gamper, J. (2004) 'Mobile ELDIT: Challenges in the Transition from an e-Learning to an m-Learning System' Proceedings of ED-MEDIA.
[30] Ogata, H., Yin C., El-Bishouty, M.M. and Yano, Y. (2010) ‘Computer supported ubiquitous learning environment for vocabulary learning’, International Journal of Learning Technology, 5(1), pp. 5-24.
[31] Keegan, D. (2005) 'The Incorporation of M-learning into Mainstream Education and Training', in mLearn 2005 - 4th World conference on mLearning. Cape Town, South Africa. Available online http://www.iadisportal.org/digital-library/the-incorporation-of-mobile-learning-into-mainstream-education-and-training. (Accessed 10 July 2018).
[32] Park, Y. (2011) 'A Pedagogical Framework for M-learning: Categorizing Educational Applications of Mobile Technologies into Four Types', International Review of Research in Open and Distance Learning, 12(2), pp. 78-102.
[33] Brown, M., Castellano, J., Hughes, E., Worth, A. (2012) 'Integration of iPads into a Japanese university English language curriculum' JALT CALL Journal, 8(3), pp. 197-209.
[34] Kukulska-Hulme, A. and Shield, L. (2008) 'An Overview of Mobile Assisted Language Learning: from content delivery to supported collaboration and interaction', ReCALL, 20(3), pp. 249–252.
[35] Miangah, T. M. and Nezarat, A. (2012) 'Mobile-Assisted Language Learning', International Journal of Parallel Emergent and Distributed Systems, 3(1), pp. 309-319.
[36] Ellis, R. (2005) Instructed second language acquisition: A literature review. Wellington, New Zealand: New Zealand Ministry of Education. Available online https://web.stanford.edu/~hakuta/Courses/Ed388%20Website/Resources/Ellis%20Instructed-second-language%20-%20latest%20version.pdf (Accessed 3rd October 2019)
[37] Wong, L.H. and Looi, C.K. (2011) 'What seams do we remove in mobile assisted seamless learning? A critical review of the literature', Computers & Education, 57(4), pp. 2364-2381.
[38] Dickey, R.J. (2001) ‘Make it a conference call: An English conversation course by telephone in South Korea’, In L.E. Henrichsen (Ed.), Distance-learning programs. Alexandria: Teachers of English to Speakers of Other Languages, Inc, pp. 51-60.
[39] Rosell-Aguilar, F. (2009) ‘Podcasting for language learning: re-examining the potential’, In Lomicka, L. and Lord, G. (Eds.) The next generation: Social networking and online collaboration in foreign language learning, pp. 13–34.
[40] JISC (2005) ‘Multimedia learning with mobile phones. Innovative Practices with Elearning. Case Studies: Anytime, any place Learning’, Available online http://www.elearning.ac.uk/innoprac/learner/southampton.html. (Accessed 2nd July, 2018).
[41] Thornton, P. and Houser, C. (2005) ‘Using mobile phones in English education in Japan’, Journal of Computer-Assisted Learning, 21(3), pp. 217-228.
[42] Tomlinson, B. (1998) ‘Introduction’, in Tomlinson, B. (ed.) Materials development in language teaching. Cambridge: Cambridge University Press, pp. 1-24.
[43] Tomlinson, B. (2001) ‘Materials development’, in Carter, R. and Nunan, D. (eds) The Cambridge Guide to Teaching English to Speakers of Other Languages. Cambridge: Cambridge University Press (The Cambridge Guides), pp. 66–71.
[44] McGrath, I. (2013) Teaching Materials and the Roles of EFL/ESL Teachers: Practice and Theory. London: Bloomsbury Publishing PLC. Available at: https://ebookcentral.proquest.com/lib/ucd/detail.action?docID=1134864 (Accessed 18 April 2018).
[45] McGrath, I. (2016) Materials Evaluation and Design for Language Teaching (2nd edn). Edinburgh: Edinburgh University Press.
[46] Hofler, D. B. (1983) ‘Approach, method, technique. A clarification’, Reading World, (23)1, pp. 71-72.
[47] Richards, J.C. and Rodgers, T.S. (2001) Approaches and Methods in Language Teaching. Cambridge: Cambridge University Press.
[48] Anthony, E. M. (1963) ‘Approach, method and technique’, English Language Teaching, 17, pp. 63-67.
[49] Arús-Hita, J., Rodríguez-Arancón, P., and Calle-Martínez, C. (2014) 'A pedagogic assessment of m-learning applications', in Proceedings of ICDE 2013, Mobilizing Distance Education. Madrid: UNED.
[50] Burston, J. (2014) 'The reality of mall: Still on the fringes'. CALICO Journal, 31(1), pp. 103–125.
[51] Stockwell, G., and Hubbard, P. (2013) Some emerging principles for mobile-assisted language learning. Monterey, CA: The International Research Foundation for English Language Education. Available
[52] Liu, M., Geurtz, R, Karam, A., Navarrete, C. and Scordino, R. (2013) ‘Research on Mobile Learning in Adult Education’, In W. Kinuthia and S. Marshall (Eds.) On the Move: Mobile Learning for Development, Charlotte: Information Age Publishing.
[53] Burston, J. (2013) Mobile assisted language learning: A selected annotated bibliography of implementation studies 1994-2012. Language Learning and Technology, 17(3), pp. 157-225.
[54] Crompton, H. (2014) 'A Diachronic overview of technology contributing to mobile learning: A Shift towards student-centred pedagogies', in Ally, M. and Tsinakos, A. (Eds.) Increasing Access through Mobile Learning. Vancouver: Commonwealth of Learning and Athabasca University, pp. 7-15.
[55] Rodgers, T. S. (2000) ‘Methodology in the new millennium’, English Teaching Forum, 38(2), pp. 2–13.
[56] Brown, H. D. (2002) ‘English Language Teaching in the post-Method era: Toward better diagnosis, treatment, and assessment’, In Richards, J. C. and Renandya, W. A. (Eds.) Methodology in language teaching: An anthology of current practice. Cambridge: CUP, pp.9-18.
[57] Kumaravadivelu, B. (1994) ‘The Post-Method Condition: (E)merging strategies for second/foreign language teaching’, TESOL Quarterly, 28(1), pp. 27-48.
[58] Kumaravadivelu, B. (2006) Understanding language teaching: From method to postmethod. New Jersey: Lawrence Erlbaum Associates.
[59] Allwright, D. and Hanks, J. (2009) The developing language learner: An introduction to exploratory practice. London: Palgrave Macmillan
[60] Littlewood, W. (2011) ‘Communicative language teaching: An expanding concept for a changing world’, In Hinkel, E. (ed.), Handbook of Research in Second Language Teaching and Learning. New York: Routledge, pp. 541-547.
[61] Clarke, M. A. (1983) ‘The scope of approach, the importance of method, and the nature of techniques’, In Alatis, J. E., Stem, H. H. and Strevens, P. (Eds.), Georgetown University round table on language and linguistics. Washington: Georgetown University Press, pp. 106-115.
[62] Pennycook, A. (1989) ‘The Concept of Method, Interested Knowledge, and the Politics of Language Teaching’, TESOL Quarterly, 23(4), pp. 589-618.
[63] Jarvis, G. A. (1991) ‘Research on teaching methodology: Its evolution and prospects’, In Feed, B. (Ed.), Foreign language acquisition research and the classroom. Lexington: D. C. Heath, pp. 295- 306.
[64] Prabhu, N. S. (1990) ‘There is no best method —Why?’ TESOL Quarterly, 24(2), pp. 161-176.
[65] Hutchinson, T. and Waters, A. (1987) English for specific purposes: A learning-centred approach. Cambridge: Cambridge University Press.
[66] Littlewood, W. (2013) ‘Developing a Context-sensitive Pedagogy for Communication-oriented Language Teaching’, English Teaching, 68(3), pp. 3-24.
[67] Doughty, C. J. and Long, M. H. (2003) 'Optimal psycholinguistic environments for distance foreign language learning', Language Learning & Technology, 7(3), pp. 50–80.
[68] Arnold, J., Dörnyei, Z. and Pugliese, C. (2015) The principled communicative approach: Seven criteria for success. London: Helbling Languages.
[69] CoE (2001) Common European Framework of Reference for Languages: Learning, Teaching, Assessment. Available online https://rm.coe.int/16802fc1bf (Accessed 17th June 2019)
[70] Piccardo, E. (2010) ‘From communicative to Action-Oriented: New perspectives for a new millennium’, TESL Ontario Research Symposium Annual Edition, 36(2), pp. 20-35.
[71] Parlement Européen (2006) ‘Compétences clés pour l'éducation et la formation tout au long de la vie, (COM(2005)0548 – C6-0375/2005 – 2005/0221(COD)’, Strasbourg. Available online http://www.europarl.europa.eu/sides/getDoc.do?pubRef=-//EP//NONSGML+TA+P6-TA-2006-0365+0+DOC+PDF+V0//FR (Accessed 8th May 2019)
[72] Cignatta, T. (2006) ‘Qui a peur des compétences (clés)? Langues étrangères, FLE et compétences (clés) en Europe: la longue voie de la définition de compétences communes à la réalisation et à l’évaluation de l’apprentissage’, Synergies Europe, 1, pp. 205-215.
[73] McLoughlin, D. (2016). Review of Theorizing and analyzing agency in second language learning: Interdisciplinary approaches, by P. Deters, X. (A.) Gao, E. R. Miller, & G. Vitanova (Eds). Applied Linguistics, 37, 442–445.
[74] Larsen-Freeman, D. (2019) ‘On Language Learner Agency: A Complex Dynamic Systems Theory Perspective’, The Modern Language Journal, 103, pp. 61-79.
[75] García, O. and Flores, N. (2014) ‘Multilingualism and common core state standards in the United States’, In May, S. (Ed.) The multilingual turn. New York: Routledge/Taylor and Francis, pp. 147–166.
[76] Kumaravadivelu, B. (2001) ‘Toward a Postmethod Pedagogy’ TESOL Quarterly, 35, pp. 537-560.
[77] CoE (2020) Common European Framework of Reference for Languages: Learning, Teaching, Assessment. Companion Volume with New Descriptors. Available online https://rm.coe.int/cefr-companion-volume-with-new-descriptors-2018/1680787989 (Accessed 17th June 2019)
[78] USNY (1987) Modern Language for Communication CoE Available online http://www.nysed.gov/common/nysed/files/programs/world-languages/modernl.pdf (Accessed 29th April 2019)
[79] Butler, Y. G. and Lino, M. (2005) ‘Current Japanese reforms in English language education: The 2003 «Action Plan»’, Language Policy, 4(1), pp. 25-45.
[80] Wang, Q. (2007) ‘The National Curriculum changes and their effects on English language teaching in the People’s Republic of China’, In Cummins, J. and Davison, C. (Eds.), International handbook of English language teaching. Boston: Springer Science & Business Media, pp. 87-105.
[81] Wang, W., and Lam, A. (2009) ‘The English Language Curriculum for Senior Secondary School in China: Its evolution from 1949’, RELC Journal, 40(1), pp. 65-82.
[82] Piccardo, E. (2018) ‘Plurilingualism: Vision, Conceptualization, and Practices’, In Trifonas, P., Aravossitas, T. (Eds). Springer International Handbooks of Education. Handbook of Research and Practice in Heritage Language Education. New York, NY: Springer International Publishing. Pp. 207-226.
[83] Runnels, J. and Runnels, V. (2019) ‘Impact of the Common European Framework of Reference—A bibliometric analysis of research from 1990-2017’, CEFR Journal, pp. 18-32.