EMOES: Eye Motion and Ocular Expression Simulator

Authors: Nicoletta Adamo-Villani, Gerardo Beni, Jeremy White

Abstract:

We introduce, a new interactive 3D simulation system of ocular motion and expressions suitable for: (1) character animation applications to game design, film production, HCI (Human Computer Interface), conversational animated agents, and virtual reality; (2) medical applications (ophthalmic neurological and muscular pathologies: research and education); and (3) real time simulation of unconscious cognitive and emotional responses (for use, e.g., in psychological research). The system is comprised of: (1) a physiologically accurate parameterized 3D model of the eyes, eyelids, and eyebrow regions; and (2) a prototype device for realtime control of eye motions and expressions, including unconsciously produced expressions, for application as in (1), (2), and (3) above. The 3D eye simulation system, created using state-of-the-art computer animation technology and 'optimized' for use with an interactive and web deliverable platform, is, to our knowledge, the most advanced/realistic available so far for applications to character animation and medical pedagogy.

Keywords: 3D animation, HCI, medical simulation, ocularmotion and expression.

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1080977

References:

[1] F. Parke, "Parameterized models of human faces". PhD Thesis, University of Utah, 1974.
[2] S. Platt and N. Badler, "Animating facial expressions". Computer Graphics (Siggraph- 81 Proc.), 1981, pp.279-288.
[3] K. Waters, "A muscle model for animating three dimensional facial expressions". Computer Graphics (Siggraph -87 Proc.), 1987, pp.17- 24.
[4] P. Kalra, A. Mangili, N. Magnenat Thalmann and D. Thalmann , "Simulation of Facial Muscle Actions Based on Rational Free Form Deformations". Computer Graphics Forum, 1992,, Vol. 11, Issue 3, pp. 59-69.
[5] S. P. Lee, J. B. Badler, and N. I. Badler. "Eyes Alive". ACM Transactions on Graphics, 2002, 21 (3), pp. 637-644.
[6] N. Adamo-Villani, G. Beni, "Keyboard Encoding of Facial Expressions". IEEE Proceedings of IV04 - 8th International Conference on Information Visualization, London, 2004, pp. 324-328.
[7] J. Cassell, H. Vilhjalmsson, and T. Bickmore, "BEAT: the Behavior Expression Animation Toolkit" Siggraph 2001 Proc.
[8] J. Cassel, O. Torees, S. Prevost, "Turn taking vs. discourse structure: how best to model multimodal conversation". In Machine Conversations, Wilks eds, pp. 143-154
[9] The Polar Express Production Notes (Online). Available: (http://polarexpressmovie.warnerbros.com/movie_prodnotes.html)
[10] Online Interactive Patient Simulators, New York Emergency Room. (http://www.nyerrn.com/simulators.htm )
[11] R. Lasslo, Eye Simulation Application. (Online). Available: (http://cim.ucdavis.edu/Eyes/version1/eyesim.htm)
[12] Eye Simulator/Virtual Patient Simulator. (Online). Available: (http://cim.ucdavis.edu/EyeRelease/Interface/TopFrame.htm)
[13] R. Cole, J. Movellan, J. Gratch. NSF Workshop proposal on Perceptive Animated Interfaces and Virtual Humans, 2002
[14] A. Bayklor, Y. Kim. "Pedagogical Agent Design: The Impact of Agent Realism, Gender, Ethnicity, and Instructional Role". Proc. of International Conference on Intelligent Tutoring Systems, 2004.
[15] K. Moreno, B. Klettke , K. Nibbaragandla, A. C. Graesser. "Perceived Characteristics and Pedagogical Efficacy of Animated Conversational Agents". Proc. of Intelligent Tutoring Systems : 6th International Conference, France, 2002.
[16] T. Cacioppo, L. Tassinary. Principles of psychophysiology: physical, social, and inferential elements. Cambridge University Press, 1990.
[17] E. Hecht. Optics. 2nd Ed. Addison Wesley, 1987.
[18] P Ekman, W Friesen, J Hager. Facial Action Coding System: A Technique for the Measurement of Facial Movement. Consulting Psychologists Press, Palo Alto, CA, 1978.
[19] E. Ackerman. Considerable material on vision from a medical point of view. Biophysical Science, Prentice-Hall, 1962.
[20] F. Delacerda. Some interesting details about accommodation of the eye. Collision Avoidance, Flight Training, May 1991.
[21] L.Stark, P.M. Sherman. A servoanalytic study of consensual pupil reflex to light. Yale University, 1956.
[22] Game Controller: Belkin Nostromo Speedpad n52. (Online). Available: (http://www.tomshardware.com/game/200403061/ )
[23] Mathsigner (Online). Available : (http://www.tech.purdue.edu/cgt/I3/)
[24] N. Adamo-Villani, J. Doublestein, Z. Martin. "The MathSigner: an Interactive Learning Tool for American Sign Language K-3 Mathematics". IEEE Proceedings of IV04 - 8th International Conference on Information Visualization, London, 2004, pp.713-716.
[25] N. Adamo-Villani, J. Doublestein, Z. Martin. "Sign Language for K-8 Mathematics by 3D Interactive Animation". Journal of Educational Technology Systems, Vol. 33, No. 3, pp. 241 - 257, 2004-2005.
[26] C. Baker and C.A. Padden. Understanding language through sign language research. Focusing on the nonmanual components of American Sign Language. Academic Press, Inc., 1978.
[27] Mather, S. A. Eye gaze and communication in a deaf classroom. Sign Language Studies, 54, pp. 11-30, 1987.
[28] R. Wilbur. Eyeblinks and ASL Phrase Structure. Sign Language Studies, 84, pp.221-240, 1994.
[29] Fakespace systems (Online). Available: (www.fakespace.com),
[30] CAL3D (Online). Available: (http://cal3d.sourceforge.net/).
[31] Open Scene Graph (Online). Available: (http://www.openscenegraph.org).
[32] Virtual Juggler (Online). Available: (http://www.vrjuggler.org ).
[33] Intersense, Inc. (Online). Available: (http://www.isense.com/)