Humanoid Personalized Avatar Through Multiple Natural Language Processing

Authors: Jin Hou, Xia Wang, Fang Xu, Viet Dung Nguyen, Ling Wu

Abstract:

There has been a growing interest in implementing humanoid avatars in networked virtual environment. However, most existing avatar communication systems do not take avatars- social backgrounds into consideration. This paper proposes a novel humanoid avatar animation system to represent personalities and facial emotions of avatars based on culture, profession, mood, age, taste, and so forth. We extract semantic keywords from the input text through natural language processing, and then the animations of personalized avatars are retrieved and displayed according to the order of the keywords. Our primary work is focused on giving avatars runtime instruction from multiple natural languages. Experiments with Chinese, Japanese and English input based on the prototype show that interactive avatar animations can be displayed in real time and be made available online. This system provides a more natural and interesting means of human communication, and therefore is expected to be used for cross-cultural communication, multiuser online games, and other entertainment applications.

Keywords: personalized avatar, mutiple natural luanguage processing, social backgrounds, anmimation, human computer interaction

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1081862

References:

[1] T. Capin, L. Pandzic, N. M. Thalmann, and D. Thalmann, Avatars in Networked Virtual Environments, John Wiley & Sons Ltd, England, 1999.
[2] H. Tang, Y. Fu, J. L. Tu, M. Hasegawa-Johnson, and T. S. Huang, "Humanoid audio-visual avatar with emotive text-to-speech synthesis," IEEE Trans. Mul., vol. 10, no. 6, pp. 969-981, October 2008.
[3] Y. Fu, R. Li, T. S. Huang, and M. Danielsen, "Real-time multimodal human-avatar interaction," IEEE Trans. CSVT, vol. 18, no. 4, pp. 467- 477, April 2008.
[4] X. L. Yang, D. C. Petriu, T. E. Whalen, and E. M. Petriu, "Hierarchical animation control of avatars in 3-D virtual environments," IEEE Trans. Instrumentation and Measurement, vol. 54, no. 3, pp. 1333-1341, June 2005.
[5] J. Hou, Y. Koine, and Y. Aoki, "Ontology-based social avatar language," Journal of Signal Processing, vol. 7, no. 5, pp. 393-403, Septemper 2003.
[6] T. Noma, L. W. Zhao, and N. Badler, "Design of a virtual human presenter," IEEE Comput. Graph. Appl., vol. 20, no. 4, pp. 79-85, July 2000.
[7] J. Cassell, H. Vilhjalmsson, and T. Bickmore, "BEAT: the behavior expression animation toolkit," in Proc. 28th Int. Conf. Computer Graphics and Interactive Techniques, Los Angeles, USA, Aug. 2001, pp. 477-486.
[8] Z. X. Yang, L. Li, and D. Zhang, "Embodiment of text based on cirtual robotic avatar," in Proc. IEEE Int. Conf. Robotics and Biomimetics, Sanya, China, Dec. 2007, pp. 1285 - 1289.
[9] W. Steptoe and A. Steed, "High-fidelity avatar eye-representation," in Proc. IEEE Virtual Reality Conf., Nevada, USA, Mar. 2008, pp. 111-114.
[10] Z. Sakr and C. Sudama, "A curvilinear avatar with avatar collision detection scheme in collaborative virtual environments," in Proc. IEEE Instrumentation and Measurement Technology Conf., Victoria Vancouver Island, Canada, May 2008, pp. 1027 - 1030.
[11] Y. Ishii and T. Watanabe, "An embodied avatar mediated communication system with VirtualActor for human interaction analysis," in Proc. 16th IEEE Int. Conf. Robot and Human Interactive Communication, Jeju, Korea, Aug. 2007, pp. 37-42.
[12] S. Piotr and K. Bozena, "Personalized avatar animation for virtual reality," in Proc. 1st Int. Conf. Information Technology, Gdansk, Poland, May 2008, pp. 1-4.
[13] S. Rusdorf, G. Brunnett, M. Lorenz, and T. Winkler, "Real-time interaction with a humanoid avatar in an immersive table tennis simulation," IEEE Trans. Visualization and Computer Graphics, vol. 13, no. 1, pp. 15-25, January-February 2007.
[14] C. Lee, H. Lee, and K. Oh, "Real-time image-based 3D avatar for immersive game." in Proc. 7th ACM SIGGRAPH Int. Conf. Virtualreality Continuum and Its Applications in Industry, Singapore, Dec. 2008, pp. 1-2.
[15] N. Ahmed, E. Aguiar, C. Theobalt, M. Magnor, and H. P. Seidel, "Automatic generation of personalized human avatars from multi-view video." in Proc. the ACM Symposium Virtual Reality Software and Technology, Monterey, USA, Nov. 2005, pp. 257-260.
[16] The Humanoid Animation Group-s. (Online). Available: http://www.hanim.org/.
[17] N. M. Thalmann, H. Seo, and F. Cordier, "Automatic modeling of animatable virtual humans-a survey," in Proc. 4th Int. Conf. 3-D Digital Imaging and Modeling, Banff, Canada, Oct. 2003, pp. 2-10.
[18] I. J. Pelczer, F. Cabiedes, and F. Gamboa, "Emotions and interactive agents," in Proc. IEEE Int. Conf. VECIMS, La Coru├▒a, Spain, Jul. 2006, pp.184-187,.
[19] Globe Warp-s web site (Online). Available: http://chat.globewarp.or.jp/index.html
[20] T. Wang, X. Li, and J. Shi, "An avatar-based approach to 3D user interface design for children," in Proc. IEEE Symposium 3D User Interfaces, North Carolina, USA, Mar. 2007, pp. 155-161,.
[21] Sony-s 3-D Chat web site (Online). Available: http://www.sonet. ne.jp/paw
[22] Y. J. Oh, K. H. Park, and Z. Bien, "Body motion editor for sign language avatar," in Proc. Int. Conf. Control, Automation and Systems, Seoul, Korea, Oct. 2007, pp. 1752-1757.