Interactive Shadow Play Animation System

Authors: Bo Wan, Xiu Wen, Lingling An, Xiaoling Ding

Abstract:

The paper describes a Chinese shadow play animation system based on Kinect. Users, without any professional training, can personally manipulate the shadow characters to finish a shadow play performance by their body actions and get a shadow play video through giving the record command to our system if they want. In our system, Kinect is responsible for capturing human movement and voice commands data. Gesture recognition module is used to control the change of the shadow play scenes. After packaging the data from Kinect and the recognition result from gesture recognition module, VRPN transmits them to the server-side. At last, the server-side uses the information to control the motion of shadow characters and video recording. This system not only achieves human-computer interaction, but also realizes the interaction between people. It brings an entertaining experience to users and easy to operate for all ages. Even more important is that the application background of Chinese shadow play embodies the protection of the art of shadow play animation.

Keywords: Gesture recognition, Kinect, shadow play animation, VRPN.

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1337941

References:

[1] Ugur Güdükbay, Fatih Erol, Nezih Erdoğan. Tradition offers artistic possibilities for new media technologies:An animation system for shadow theatre, actes / proceedings isea2000 – 10/12/2000 – salle/room 300.
[2] Yi-Bo Zhu, Chen-Jia Li, I-Fan Shen, Kwan-Liu Ma, Aleksander Stompel, A New Form of Traditional Art – Visual Simulation of Chinese Shadow Play. Proceeding SIGGRAPH '03, ACM SIGGRAPH 2003 Sketches & Applications. 2003:1-1.
[3] GAO Lu-jing, CAI jian-ping, Analysis and Implementation of Features of Character’s Action in Shadowgraph Animation. Computer Engineering and Design, 2010. 2335-2342
[4] S.W. Hsu, T.Y. Li. Generating Secondary Motions in Shadow Play Animations with Motion Planning Techniques. Proceedings of SIGGRAPH 2005 conference on Sketches & applications, 2005, Article No. 69,1 pages.
[5] S.W. Hsu, T.Y. Li. Planning Character Motions for Shadow Play Animations. Proceedings of Computer Animation and Social Agents, 2005:1-6.
[6] Totilo, Stephen (January 7, 2010). "Natal Recognizes 31 Body Parts, Uses Tenth of Xbox 360 "Computing Resources"". Kotaku, Gawker Media. Retrieved November 25, 2010. http://en.wikipedia.org/wiki/Kinect#Technology
[7] L. Leite and V. Orvalho, “Shape your body: control a virtual silhouette using body motion” CHI '12 Extended Abstracts on Human Factors in Computing Systems: 1913-1918
[8] Seth Hunter, Pattie Maes. Designing digital puppetry systems: guidelines and best practices. Extended Abstracts on Human Factors in Computing Systems. 2013
[9] Robert Held, Ankit Gupta, Brian Curless, Maneesh Agrawala. 3D puppetry: a kinect-based interface for 3D animation. Proceedings of the 25th annual ACM symposium on User interface software and technology, Oct., 2012.
[10] Bradley Wesson, Brett Wilkinson. Evaluating organic 3D sculpting using natural user interfaces with the Kinect, Proceedings of the 25th Australian Computer-Human Interaction Conference, Nov., 2013.
[11] Kam Lai, Konrad, J., Ishwar, P. A gesture-driven computer interface using Kinect, published byImage Analysis and Interpretation (SSIAI), 2012 IEEE. Pages: 185 – 188.
[12] Karthik Nandakumar, Kong Wah Wan, Siu Man Alice Chan, Wen Zheng Terence Ng, Jian Gang Wang, Wei Yun Yau. A multi-modal gesture recognition system using audio, video, and skeletal joint data, Proceedings of the 15th ACM on International conference on multimodal interaction, Dec.,2013
[13] Tsai-Te Chu, Chung-Yen Su. A Kinect-based Handwritten Digit Recognition for TV Remote Controller, published byIntelligent Signal Processing and Communications, 4-7 Nov. 2012.Pages: 414 – 419
[14] Anderson, D., Bailey, C. and Skubic, M. (2004) Hidden Markov Model symbol recognition for sketch-based interfaces.AAAI Fall Symposium. Menlo Park, CA: AAAI Press, 15-21.
[15] M. Black, A. Jepson. A probabilistic framework for matching temporal trajectories: Condensation-based recognition of gestures and expressions. in Proc. European Conf. on Computer Vision(ECCV’98), 1998
[16] M. J. Black, A. D. Jepson. Recognition temporal trajectories using the Condensation algorithm. in Proc. IEEE Int’1 Conf. on Automated Face and Gesture Recognition(FG’98), Japan, 1998: 16~21
[17] Asma Ben Hadj Mohamed, Thierry VAL, Laurent Andrieux, Abdennaceur KACHOUR. Assisting people with disabilities through Kinect sensors into a smart house, International Conference on Computer Medical Application,Jan.2013.Pages:1-5
[18] Foley, J., V.L. Wallace, and P. Chan,The Human Factors of Computer Graphics Interaction Techniques. IEEE Computer Graphics and Application, 1984.4 (11): p. 13-48.
[19] Russell M. Taylor II, Thomas C. Hudson, Adam Seeger, Hans Weber, Jeffrey Juliano, Aron T. Helser. VRPN: A Device-Independent, Network-Transparent VR Peripheral System. Proceedings of VRST 2001, ACM Virtual Reality Software and Technology. 2001.
[20] http://depts.washington.edu/aimgroup/proj/dollar/
[21] Jacob O.Wobbrock, Andrew D.Wilson and Yang Li.(2007) Gestures without libraries, toolkits or training: a $1 recognizer for user interface prototypes. Proc. UIST ’07. New York: ACM Press, 159-168.
[22] Paul Doliotis, Alexandra Stefan, Christopher McMurrough, David Eckhard, Vassilis Athitsos. Comparing gesture recognition accuracy using color and depth information, Proceedings of the 4th International Conference on PErvasive Technologies Related to Assistive Environments, May, 2011.
[23] Jaemin Soh, Yeongjae Choi, Youngmin Park, Hyun S. Yang. User-friendly 3D object manipulation gesture using Kinect, Nov., 2013.