3D Simulator of Ocular Motion and Expression

Authors: Nicoletta Adamo-Villani, Gerardo Beni, Jeremy White

Abstract:

We introduce a new interactive 3D simulator of ocular motion and expressions suitable for: (1) character animation applications to game design, film production, HCI (Human Computer Interface), conversational animated agents, and virtual reality; (2) medical applications (ophthalmic neurological and muscular pathologies: research and education); and (3) real time simulation of unconscious cognitive and emotional responses (for use, e.g., in psychological research). Using state-of-the-art computer animation technology we have modeled and rigged a physiologically accurate 3D model of the eyes, eyelids, and eyebrow regions and we have 'optimized' it for use with an interactive and web deliverable platform. In addition, we have realized a prototype device for realtime control of eye motions and expressions, including unconsciously produced expressions, for application as in (1), (2), and (3) above. The 3D simulator of eye motion and ocular expression is, to our knowledge, the most advanced/realistic available so far for applications in character animation and medical pedagogy.

Keywords: 3D animation, HCI, medical simulation, ocularmotion and expression.

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1332608

References:

[1] F. Parke, "Parameterized models of human faces". PhD Thesis, University of Utah, 1974.
[2] S. Platt and N. Badler, "Animating facial expressions". Computer Graphics (Siggraph- 81 Proc.), 1981, pp.279-288.
[3] K. Waters, "A muscle model for animating three dimensional facial expressions". Computer Graphics (Siggraph -87 Proc.), 1987, pp.17- 24.
[4] P. Kalra, A. Mangili, N. Magnenat Thalmann and D. Thalmann, "Simulation of Facial Muscle Actions Based on Rational Free Form Deformations". Computer Graphics Forum, 1992, Vol. 11, Issue 3, pp. 59-69.
[5] S. P. Lee, J. B. Badler, and N. I. Badler. "Eyes Alive". ACM Transactions on Graphics, 2002, 21 (3), pp. 637-644.
[6] N. Adamo-Villani, G. Beni, "Keyboard Encoding of Facial Expressions". IEEE Proceedings of IV04 - 8th International Conference on Information Visualization, London, 2004, pp. 324-328.
[7] J. Cassell, H. Vilhjalmsson, and T. Bickmore, "BEAT: the Behavior Expression Animation Toolkit" Siggraph 2001 Proc.
[8] J. Cassel, O. Torees, S. Prevost, "Turn taking vs. discourse structure: how best to model multimodal conversation". In Machine Conversations, Wilks eds, pp. 143-154.
[9] The Polar Express Production Notes (Online). Available: http://polarexpressmovie.warnerbros.com/movie_prodnotes.html
[10] Online Interactive Patient Simulators, New York Emergency Room. http://www.nyerrn.com/simulators.htm
[11] R. Lasslo, Eye Simulation Application. (Online). Available: http://cim.ucdavis.edu/Eyes/version1/eyesim.htm
[12] Eye Simulator/Virtual Patient Simulator. (Online). Available: http://cim.ucdavis.edu/EyeRelease/Interface/TopFrame.htm
[13] R. Cole, J. Movellan, J. Gratch. NSF Workshop proposal on Perceptive Animated Interfaces and Virtual Humans, 2002.
[14] A. Bayklor, Y. Kim. "Pedagogical Agent Design: The Impact of Agent Realism, Gender, Ethnicity, and Instructional Role". Proc. of International Conference on Intelligent Tutoring Systems, 2004.
[15] K. Moreno, B. Klettke , K. Nibbaragandla, A. C. Graesser. "Perceived Characteristics and Pedagogical Efficacy of Animated Conversational Agents". Proc. of Intelligent Tutoring Systems : 6th International Conference, France, 2002.
[16] T. Cacioppo, L. Tassinary. Principles of psychophysiology: physical, social, and inferential elements. Cambridge University Press, 1990.
[17] E. Hecht. Optics. 2nd Ed. Addison Wesley, 1987.
[18] P Ekman, W Friesen, J Hager. Facial Action Coding System: A Technique for the Measurement of Facial Movement. Consulting Psychologists Press, Palo Alto, CA, 1978.
[19] E. Ackerman. Considerable material on vision from a medical point of view. Biophysical Science, Prentice-Hall, 1962.
[20] F. Delacerda. Some interesting details about accommodation of the eye. Collision Avoidance, Flight Training, May 1991.
[21] L.Stark, P.M. Sherman. A servoanalytic study of consensual pupil reflex to light. Yale University, 1956.
[22] Game Controller: Belkin Nostromo Speedpad n52. (Online). Available: http://www.tomshardware.com/game/200403061/
[23] Mathsigner (Online). Available :http://www.tech.purdue.edu/cgt/I3/
[24] N. Adamo-Villani, J. Doublestein, Z. Martin. "The MathSigner: an Interactive Learning Tool for American Sign Language K-3 Mathematics". IEEE Proceedings of IV04 - 8th International Conference on Information Visualization, London, 2004, pp.713-716.
[25] N. Adamo-Villani, J. Doublestein, Z. Martin. "Sign Language for K-8 Mathematics by 3D Interactive Animation". Journal of Educational Technology Systems, Vol. 33, No. 3, pp. 241 - 257, 2004-2005.
[26] Fakespace systems (Online). Available: www.fakespace.com
[27] CAL3D (Online). Available: http://cal3d.sourceforge.net/
[28] Open Scene Graph (Online). Available: http://www.openscenegraph.org
[29] Virtual Juggler (Online). Available: http://www.vrjuggler.org
[30] Intersense, Inc. (Online). Available: http://www.isense.com/