An AR/VR Based Approach Towards the Intuitive Control of Mobile Rescue Robots

Authors: Jürgen Roßmann, André Kupetz, Roland Wischnewski

Abstract:

An intuitive user interface for the teleoperation of mobile rescue robots is one key feature for a successful exploration of inaccessible and no-go areas. Therefore, we have developed a novel framework to embed a flexible and modular user interface into a complete 3-D virtual reality simulation system. Our approach is based on a client-server architecture to allow for a collaborative control of the rescue robot together with multiple clients on demand. Further, it is important that the user interface is not restricted to any specific type of mobile robot. Therefore, our flexible approach allows for the operation of different robot types with a consistent concept and user interface. In laboratory tests, we have evaluated the validity and effectiveness of our approach with the help of two different robot platforms and several input devices. As a result, an untrained person can intuitively teleoperate both robots without needing a familiarization time when changing the operating robot.

Keywords: Teleoperation of mobile robots, augmented reality, user interface, virtual reality.

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1077479

References:

[1] E. Freund and J. Rossmann, "Projective Virtual Reality: Bridging the gap between Virtual Reality and Robotics," in IEEE Transactions on Robotics and Automation, Vol. 15, No. 3, pages 411-422, June 1999
[2] J. Roßmann, P. Krahwinler, and C. Schlette, "Navigation of mobile robots in natural environments: Using sensor fusion in forestry," in The 2nd International Multi-Conference on Engineering and Technological Innovation (IMETI), Orlando, USA, Vol. 3, pages 5-9, July 2009.
[3] J. Roßmann, M. Schluse, and C. Schlette, "The virtual forest: Robotics and simulation technology as the basis for new approaches to the biological and the technical production in the forest," in The 13th World Multi-Conference on Systemics, Cybernetics and Informatics: WMSCI 2009, Orlando, USA, Vol. 2, pp. 33-38, July 2009.
[4] S. Jeschke, T. Aach, L. Eckstein, J. Rossmann, and B. Rumpe, "RescueCars - Autonomous Service Vehicles in Rescue Scenarios," RWTH Aachen University, Germany, Tech. Rep., Feb. 2010.
[5] J. Casper and R.R. Murphy, "Human-Robot Interactions During the Robot-Assisted Urban Search and Rescue Response at the World Trade Center," in IEEE Transactions on Systems, Man, and Cybernetics, Vol. 33, No. 3, pages 367-385, Jun. 2003.
[6] A. Davids, "Urban Search and Rescue Robots: from Tragedy to Technology," in IEEE Intelligent Systems, Vol. 17, Issue 2, pages 81- 83, Mar. 2002.
[7] http://www.robocuprescue.org/, visited Jun. 2010.
[8] M. Montemerlo, S. Thrun, D. Kooler, and B. Wegbreit, "FastSLAM: A Factored Solution to the Simultaneous Localization and Mapping Problem," in Proceedings of the AAAI National Conference on Artificial Intelligence, Edmonton, Canada, 2002.
[9] J. Suthakorn, S.S.H. Shah, S. Jantarajit, W. Onprasert, et al., "On the Design and Development of A Rough Terrain Robot for Rescue Missions," in Proceedings of the IEEE International Conference on Robotics and Biomimetics, Bangkok, Thailand, pages 1830-1835, Feb. 2009.
[10] M. Wang and J.N.K. Lui, "A Novel Teleoperation Paradigm for Human-Robot Interaction," in Proceedings of the 2004 IEEE Conference on Robotics, Automation and Mechatronics, Singapore, pages 13-18, Dec. 2004.
[11] H.A. Yanco and J. Drury, "-Where am I?- Acquiring situation awareness using a remote robot platform," in IEEE International Conference on Systems, Man and Cybernetics, Vol. 3, pages 2835- 2840, Oct. 2004
[12] M. Backer, R. Casey, B. Keyes, and H.A. Yanco, "Improved Interfaces for Human-Robot Interaction in Urban Search and Rescue," in IEEE International Conference on Systems, Man and Cybernetics, Vol. 3, pages 2960-2965, Oct. 2004.
[13] J.L. Durry, J. Scholz, and H.A. Yanco, "Applying CSCW and HCI techniques to human-robot interaction," in Proceedings CHI 2004 Workshop on Shaping Human-Robot interaction, Vieanna, USA, pages 13-16, Apr. 2004.
[14] Y.D. Kim, Y.G. Kim, S.H Lee, J.H. Kang, et al., "Portable Fire Evacuation Guide Robot System," in The 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems, St. Louis, USA, pages 2789-2794, Oct. 2009.
[15] http://www.bluetooth.com/Bluetooth/Technology/, visited Jun. 2010.
[16] R. Muraui, K. Ito, and F. Matsuno, "An intuitive Human-Robot Interface for Rescue Operation of a 3D Snake Robot the Intuitive Human-Robot Interface Aims for Non-Professional Operators" in Proceedings of the 12th IASTED International Conference Robotics and Applications, Honolulu, USA, pages 138-143, Aug. 2006.
[17] D. Tamura, A. Fujino, and K. Ito, "A Rescue Robot designed for ease of Use: Development of Exploration System using Behavior of Bombycid," in Artificial life and Robotics, Vol. 13, No. 1, pages 251- 254, Dec. 2008.
[18] G. E. Krasner and S. T. Pope, "A cookbook for using the model-view controller user interface paradigm in Smalltalk-80," in Journal of Object-Oriented Programming, Vol. 1, Issue 3, pages 26-49, Sept. 1988.
[19] http://www.festo-didactic.com/int-en/learning-systems/education-andresearch- robots-robotino/ visited Jun. 2010.