3D Spatial Interaction with the Wii Remote for Head-Mounted Display Virtual Reality

Authors: Yang-Wai Chow

Abstract:

This research investigates the design of a low-cost 3D spatial interaction approach using the Wii Remote for immersive Head-Mounted Display (HMD) virtual reality. Current virtual reality applications that incorporate the Wii Remote are either desktop virtual reality applications or systems that use large screen displays. However, the requirements for an HMD virtual reality system differ from such systems. This is mainly because in HMD virtual reality, the display screen does not remain at a fixed location. The user views the virtual environment through display screens that are in front of the user-s eyes and when the user moves his/her head, these screens move as well. This means that the display has to be updated in realtime based on where the user is currently looking. Normal usage of the Wii Remote requires the controller to be pointed in a certain direction, typically towards the display. This is too restrictive for HMD virtual reality systems that ideally require the user to be able to turn around in the virtual environment. Previous work proposed a design to achieve this, however it suffered from a number of drawbacks. The aim of this study is to look into a suitable method of using the Wii Remote for 3D interaction in a space around the user for HMD virtual reality. This paper presents an overview of issues that had to be considered, the system design as well as experimental results.

Keywords: 3D interaction, head-mounted display, virtual reality, Wii remote

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1054992

References:

[1] M. Katzourin, D. Ignatoff, L. Quirk, J.J. LaViola Jr. and O.C. Jenkins, "Swordplay: Innovating game development through VR," IEEE Computer Graphics and Applications, vol. 26, no. 6, 2006, pp. 15-18.
[2] D.A. Bowman, J. Chen, C.A. Wingrave, J. Lucas, A. Ray, N.F. Polys, Q. Li, Y. Haciahmetoglu, J.S. Kim, S. Kim, S. Boehringer and T. Ni, "New directions in 3D user interfaces," The International Journal of Virtual Reality, vol. 5, no. 2, pp. 3-14, 2006.
[3] S. Seedharam, E.S. Zurita and B. Plimmer, "3D input for 3D worlds," in Proc. of Australasian Computer-Human Interaction Conference (OzCHI 2007), Adelaide, Australia, 2007, pp. 227-230.
[4] J.J. LaViola Jr., "Bringing VR and spatial 3D interaction to the masses through video games," IEEE Computer Graphics and Applications, vol. 28, no. 5, 2008, pp. 10-15.
[5] J.C. Lee, "Hacking the Nintendo Wii Remote," Pervasive Computing, vol. 7, no. 3, 2008, pp. 39-45.
[6] WiiLi.org, http://www.wiili.org/
[7] Wiibrew.org, http://wiibrew.org/
[8] J.C. Lee, http://www.cs.cmu.edu/~johnny/projects/wii/
[9] B. Peek, http://blogs.msdn.com/coding4fun/archive/2007/03/14/ 1879033.aspx
[10] T. Schlömer, B. Poppinga, N. Henze and S. Boll, "Gesture recognition with a Wii controller," in Proc. of the 2nd International Conference on Tangible and Embedded Interaction (TEI- 08), Bonn, Germany, 2008, pp. 11-14.
[11] S.J. Castellucci, I.S. MacKenzie, "UniGest: Text entry using three degrees of motion," in Proc. of ACM CHI 2008, Florence, Italy, 2008, pp. 3549-3554.
[12] M. Lapping-Carr, O.C. Jenkins, D.H. Grollman, J.N. Schowertfeger and T.R. Hinkle, "Wiimote interfaces for lifelong robot learning," in Proc. of AAAI Symposium on using AI to Motivate Greater Participation in Computer Science, Palo Alto, C.A., 2008.
[13] A. Shirai, E. Geslin and S. Richir, "WiiMedia: Motion analysis methods and applications using a consumer video game controller," in Proc. of ACM SIGGRAPH Sandbox Symposium 2007, San Diego, C.A., 2007, pp. 133-140.
[14] L.Gallo, G.D. Pietro and I. Marra, "3D interaction with volumetric medical data: experiencing the Wiimote." in Proc. of the 1st International Conference on Ambient Medai and Systems, Quebec, Canada, 2008, article no. 14.
[15] M. Tamai, W. Wu, K. Nahrstedt, K. Yasumoto, "A view control interface for 3D tele-immersive environments," in Proc. of IEEE International Conference on Multimedia and Expo, Hannover, Germany, 2008, pp. 1101-1104.
[16] B. Bruegge, C. Teschner, P. Lachenmaier, E. Fenzl, D. Schmidt and S. Bierbaum, "Pinocchio: Conducting a virtual symphony orchestra," in Proc. of the International Conference of Advances in Computer Entertainment Technology (ACE), Salzburg, Austria, 2007, pp. 294- 295.
[17] H.J. Lee, H. Kim, G. Gupta and A. Mazalek, "WiiArts: Creating collaborative art experience with WiiRemote Interaction," in Proc. of the 2nd International Conference on Tangible and Embedded Interaction (TEI-08), Bonn, Germany, 2008, pp. 33-36.
[18] S.Hay, J. Newman and R. Harle, "Optical tracking using commodity hardware," in IEEE International Symposium on Mixed and Augmented Reality 2008, Cambridge, UK, 2008, pp. 159-160.
[19] L. Deligiannidis and J. Larkin, "Navigating inexpensively and wirelessly," in Proc. of Human System Interaction (HSI-08), Krakow, Poland, 2008, pp. 165-169.
[20] S. Attygalle, M. Duff, T. Rikakis and J. He, "Low-cost, at-home assessment system with Wii Remote based motion capture," in Proc. of Virtual Rehabilitation 2008, Vancouver, Canada, 2008, pp. 168-174.
[21] T. Schou and H.J. Gardner, "A Wii Remote, a game engine, five sensor bars and a virtual reality theatre," in Proc. of Australasian Computer- Human Interaction Conference (OzCHI 2007), Adelaide, Australia, 2007, pp. 231-234.
[22] P. Bourke, http://local.wasp.uwa.edu.au/~pbourke/projection/Wii/
[23] Nintendo, http://www.nintendo.com/wii/what/controllers
[24] H. Godbersen, "Virtual environments for anyone," IEEE Multimedia, vol. 15, no. 3, 2008, pp. 90-95.
[25] Y.W. Chow, "The Wii Remote as an input device for 3D interaction in immersive head-mounted display virtual reality," in Proc. IADIS International Conference Gaming 2008: Design for Engaging Experience and Social Interaction, Amsterdam, The Netherlands, 2008, pp. 85-92.
[26] J.J. LaViola, "A discussion of cybersickness in virtual environment," ACM SIGCHI Bulletin, vol. 32, no. 1, pp. 47-56.
[27] D. A. Bowman, E. Kruijff, J.J. LaViola Jr. and I. Poupyrev, 3D User Interfaces: Theory and Practice, Addison Wesley, 2004.
[28] G. Welch and E. Foxlin, "Motion tracking: No silver bullet, but a respectable arsenal," IEEE Computer Graphics and Applications, vol. 22, no. 6, 2002, pp. 24-38.
[29] Nintendo, http://e3.nintendo.com/pressrelease/
[30] C.A. Wingrave, D.A. Bowman and N. Ramakrishnan, "Towards preference in virtual environment interfaces," in Proc. of the Eurographics Workshop on Virtual Environments, Barcelona, Spain, 2002, pp. 63-72.
[31] G. Welch, G. Bishop, L. Vicci, S. Brumback, K. Keller and D. Colucci, "High-performance wide-area optical tracking: the HiBall tracking system," Presence: Teleoperators and Virtual Environments, vol. 10, no. 1, pp. 1-21.
[32] B. Rasco, http://www.gamasutra.com/view/feature/1494/ wheres_the_wiimote_using_kalman_.php