A Universal Approach for the Intuitive Control of Mobile Robots using an AR/VR-based Interface

Authors: Juergen Rossmann, Andre Kupetz, Roland Wischnewski

Abstract:

Mobile robots are used in a large field of scenarios, like exploring contaminated areas, repairing oil rigs under water, finding survivors in collapsed buildings, etc. Currently, there is no unified intuitive user interface (UI) to control such complex mobile robots. As a consequence, some scenarios are done without the exploitation of experience and intuition of human teleoperators. A novel framework has been developed to embed a flexible and modular UI into a complete 3-D virtual reality simulation system. This new approach wants to access maximum benefits of human operators. Sensor information received from the robot is prepared for an intuitive visualization. Virtual reality metaphors support the operator in his decisions. These metaphors are integrated into a real time stereo video stream. This approach is not restricted to any specific type of mobile robot and allows for the operation of different robot types with a consistent concept and user interface.

Keywords: 3-D simulation system, augmented reality, teleoperation of mobile robots, user interface.

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1080834

References:

[1] E. Freund and J. Rossmann, "Projective Virtual Reality: Bridging the gap between Virtual Reality and Robotics," in IEEE Transactions on Robotics and Automation, Vol. 15, No. 3, pages 411-422, June 1999.
[2] J. Rossmann, P. Krahwinler, and C. Schlette, "Navigation of mobile robots in natural environments: Using sensor fusion in forestry," in The 2nd International Multi-Conference on Engineering and Technological Innovation (IMETI), Orlando, USA, Vol. 3, pages 5-9, July 2009.
[3] J. Rossmann, M. Schluse, and C. Schlette, "The virtual forest: Robotics and simulation technology as the basis for new approaches to the biological and the technical production in the forest," in The 13th World Multi-Conference on Systemics,Cybernetics and Informatics (WMSCI ), Orlando, USA, Vol. 2, pages 33-38, July 2009.
[4] S. Jeschke, T. Aach, L. Eckstein, J. Rossmann, and B. Rumpe, "RescueCars - Autonomous Service Vehicles in Rescue Scenarios," RWTH Aachen University, Germany, Tech. Rep., Feb. 2010.
[5] J. Casper and R. R. Murphy, "Human-Robot Interactions During the Robot-Assisted Urban Search and Rescue Response at the World Trade Center," in IEEE Transactions on Systems, Man, and Cybernetics, Vol. 33, No. 3, pages 367-385, June 2003.
[6] A. Davids, "Urban Search and Rescue Robots: from Tragedy to Technology," in IEEE Intelligent Systems, Vol. 17, Issue 2, pages 81-83, Mar. 2002.
[7] http://www.robocuprescue.org/, visited May 2010.
[8] M. Montemerlo, S. Thrun, D. Kooler, and B. Wegbreit, "FastSLAM: A Factored Solution to the Simultaneous Localization and Mapping Problem," in Proceedings of the AAAI National Conference on Artificial Intelligence, Edmonton, Canada, 2002.
[9] J. Suthakorn, S. S. H. Shah, S. Jantarajit, W. Onprasert, et al., "On the Design and Development of A Rough Terrain Robot for Rescue Missions," in Proceedings of the IEEE International Conference on Robotics and Biomimetics (ROBIO), Bangkok, Thailand, pages 1830- 1835, Feb. 2009.
[10] M. Wang and J. N. K. Lui, "A Novel Teleoperation Paradigm for Human-Robot Interaction," in Proceedings of the IEEE Conference on Robotics, Automation and Mechatronics (RAM), Singapore, pages 13- 18, Dec. 2004.
[11] H. A. Yanco and J. Drury, "-Where am I?- Acquiring situation awareness using a remote robot platform," in IEEE International Conference on Systems, Man and Cybernetics (SMC), Vol. 3, pages 2835-2840, Oct. 2004.
[12] M. Backer, R. Casey, B. Keyes, and H. A. Yanco, "Improved Interfaces for Human-Robot Interaction in Urban Search and Rescue," in IEEE International Conference on Systems, Man and Cybernetics (SMC), Vol. 3, pages 2960-2965, Oct. 2004.
[13] J. L. Durry, J. Scholz, and H. A. Yanco, "Applying CSCW and HCI techniques to human-robot interaction," in Proceedings CHI 2004 Workshop on Shaping Human-Robot Interaction, Vienna, Austria, pages 13-16, Apr. 2004.
[14] Y. D. Kim, Y. G. Kim, S. H. Lee, J. H. Kang, et al., "Portable Fire Evacuation Guide Robot System," in The IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), St. Louis, USA, pages 2789-2794, Oct. 2009.
[15] R. Murai, K. Ito, and F. Matsuno, "An intuitive Human-Robot Interface for Rescue Operation of a 3D Snake Robot - The Intuitive Human-Robot Interface Aims for Non-Professional Operators" in Proceedings of the 12th IASTED International Conference on Robotics and Applications (RA), Honolulu, USA, pages 138-143, Aug. 2006.
[16] D. Tamura, A. Fujino, and K. Ito, "A Rescue Robot designed for Ease of Use: Development of Exploration System using Behavior of Bombycid," in Artificial Life and Robotics (AROB), Vol. 13, No. 1, pages 251-254, Dec. 2008.
[17] J. Borenstein, M. Hansen, and H. Nguyen, "The OmniTread OT-4 Serpentine Robot for Emergencies and Hazardous Environments," International Joint Topical Meeting: Sharing Solutions for Emergencies and Hazardous Environments, Salt Lake City, Utah, USA, Feb. 12-15, 2006.
[18] J. Baker and J. Borenstein, "The Joysnake - A Haptic Operator Console for High-Degree-of-Freedom Robots," International Joint Topical Meeting: Sharing Solutions for Emergencies and Hazardous Environments, Salt Lake City, Utah, USA, Feb. 12-15, 2006.
[19] J. Rossmann, H. Ruf, and C. Schlette, "Model-based programming 'by Demonstration' - Fast setup of robot systems (ProDemo)," in T. Kröger and F. M. Wahl (Eds.): "Advances in Robotics Research - Theory, Implementation, Application", German Workshop on Robotics, TU Braunschweig, Germany, pages 159-168, June 2009.
[20] Festo Robotino Manual, http://www.festo-didactic.com/ov3/ media/customers/1100/544305_robotino_deen.pdf, visited May 2010.