Complexity of Mathematical Expressions in Adaptive Multimodal Multimedia System Ensuring Access to Mathematics for Visually Impaired Users

Authors: Ali Awde, Yacine Bellik, Chakib Tadj

Abstract:

Our adaptive multimodal system aims at correctly presenting a mathematical expression to visually impaired users. Given an interaction context (i.e. combination of user, environment and system resources) as well as the complexity of the expression itself and the user-s preferences, the suitability scores of different presentation format are calculated. Unlike the current state-of-the art solutions, our approach takes into account the user-s situation and not imposes a solution that is not suitable to his context and capacity. In this wok, we present our methodology for calculating the mathematical expression complexity and the results of our experiment. Finally, this paper discusses the concepts and principles applied on our system as well as their validation through cases studies. This work is our original contribution to an ongoing research to make informatics more accessible to handicapped users.

Keywords: Adaptive system, intelligent multi-agent system, mathematics for visually-impaired users.

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1076938

References:

[1] Awde, A., et al., Task Migration in a Pervasive Multimodal Multimedia Computing System for Visually-Impaired Users, in 2nd International Conference on Grid and Pervasive Computing. 2007, Lecture Notes in springer-Verlag: Paris, France.
[2] Stöger, B., K. Miesenberger, and M. Batusic, Mathematical Working Environment for the Blind Motivation and Basic Ideas, in ICCHP. 2004, Springer. p. 656-663.
[3] Edwards, A.D.N. and R.D. Stevens. A Multimodal Interface for Blind Mathematics Students. in INSERM. 1994. Paris, France.
[4] Cahill, H., et al., Ensuring Usability in MATHS, in The European Context for Assistive Technology. 1995, IOS Press: Amsterdam. p. 66- 69.
[5] Preddy, M., et al. Dotsplus: How-to Make Tactile Figures and Tactile Formatted Math. in International Conference on Technology and Persons with Disabilities. 1997. Los Angeles, CA, USA.
[6] Podevin, A., Accés aux formules mathématiques par des personnes non voyantes : étude et définition d'une méthode adaptée. 2002, Université de CAEN.
[7] Ferreira, H. and D. Freitas. Enhancing the Accessibility of Mathematics for Blind People: The AudioMath Project. in ICCHP - 9th International Conference on Computer Helping People with Special Needs. 2004. Paris, France.
[8] Ferreira, H. and D. Freitas. AudioMath: Towards Automatic Readings of Mathematical Expressions. in Human-Computer Interaction International (HCII). 2005. Las Vegas, Nevada, USA.
[9] Moço, V. and D. Archabault. VICKIE: A Transcription Tool for Mathematical Braille. in AAATE -Association for the Advancement of Assistive Technology in Europe. 2003. Dublin, Ireland.
[10] Schwebel, F. and R. Goiffon. BraMaNet: Quelques règles simples ├á conna├«tre pour qu'un aveugle puisse lire vos documents mathématiques et vos pages web. in Journées nationales Caen. 2005. Caen, France.
[11] Batusic, M., K. Miesenberger, and B. Stoger. Labradoor, A Contribution To Making Mathematics Accessible For The Blind. in International Conference on Computers Helping People with Special Needs. 1998. Oldenbourg, Wien, M├╝nchen.
[12] Karshmer, A.I., et al., Reading and writing mathematics: the MAVIS project, in Proceedings of the third international ACM conference on Assistive technologies. 1998, ACM Press. p. 136-143.
[13] Nemeth, A., The Nemeth Braille Code for Mathematics and Science Notation. American Printing House for the Blind, 1972.
[14] Epheser, H., D. Pograniczna, and K. Britz, Internationale Mathematikschrift fur Blinde, Deutsche Blindenstudienanstalt. Marburg (Lahn), 1992.
[15] Commission Evolution du Braille Francais. Notation Mathematique Braille, mise à jour de la notation mathematique de 1971. 2001.
[16] Garlini, P. and F. Fogarolo, Linear Access to Mathematics for Braille Device and Audio Synthesis - Analysis of the User's Needs, in Lambda Project. 2003, University of Padova: Padova, Italy.
[17] Raman, T.V., Audio System for Technical Readings. 1st edition ed. Vol. 1410. 1999, Secaucus, NJ, USA: Springer-Verlag New York, Inc. 137.
[18] Rousseau, C., et al., A Framework for the Intelligent Multimodal Presentation of Information. Signal Processing Journal, 2006. 86(12, Special Issue on Multimodal Interface): p. 3696 - 3713.
[19] Demeure, A. and G. Calvary. Le Modèle d-Evolution en Plasticité des Interfaces: Apport des Graphes Conceptuels. in 15ème Conférence francophone sur l'Interaction Homme-Machine (IHM 2003). 2006. Caen, France: ACM Press.
[20] Awde, A., et al., Information Access in a Multimodal Multimedia Computing System for Mobile Visually-Impaired Users, in ISIE 2006, IEEE International Symposium on Industrial Electronics. 2006: Montreal, Canada.
[21] Awde, A., et al., Task Migration in a Pervasive Multimodal Multimedia Computing System for Visually-Impaired Users, in 2nd International Conference on Grid and Pervasive Computing (GPC 2007). 2007, Lecture Notes in springer-Verlag: Paris, France. p. 459-471.
[22] Bellik, Y., Interfaces multimodales : concepts, modéles et architectures., in LIMSI. 1995, Université d'Orsay: Paris.
[23] Awde, A., et al., An Adaptive Multimodal Multimedia Computing System for Presentation of Mathematical Expressions to Visually- Impaired Users, in Journal of Multimedia (JMM). to be published.
[24] Weiss, G., Multiagent systems. MIT-Press, 1999.
[25] Wooldridge, M., An Introduction to Multi-agent Systems. 2002, Chichester, UK: Wiley.
[26] Jennings, N.R. and M.J. Wooldridge, Applications of Intelligent Agents, in Agent Technology: Foundations, Applications, and Markets, N.R. Jennings and M.J. Wooldridge, Editors. 1998, Springer-Verlag: Heidelberg, Germany. p. 3-28.
[27] Awde, A., et al., A Paradigm of a Pervasive Multimodal Multimedia Computing System for the Visually-Impaired Users, in The First International Conference on Grid and Pervasive Computing. 2006: Tunghai University, Taichung, Taiwan.
[28] Cormen, T.H., et al., Introduction to Algorithms. second ed. 2001: MIT Press and McGraw-Hill.
[29] McCowan, I., et al., On the Use of Information Retrieval Measures for Speech Recognition Evaluation, in IDIAP Reseach Report. 2005.