Evaluating the Tracking Abilities of Microsoft HoloLens-1 for Small-Scale Industrial Processes

Authors: Kuhelee Chandel, Julia Åhlén, Stefan Seipel

Abstract:

This study evaluates the accuracy of Microsoft HoloLens (Version 1) for small-scale industrial activities, comparing its measurements to ground truth data from a Kuka Robotics arm. Two experiments were conducted to assess its position-tracking capabilities, revealing that the HoloLens device is effective for measuring the position of dynamic objects with small dimensions. However, its precision is affected by the velocity of the trajectory and its position within the device's field of view. While the HoloLens device may be suitable for small-scale tasks, its limitations for more complex and demanding applications requiring high precision and accuracy must be considered. The findings can guide the use of HoloLens devices in industrial applications and contribute to the development of more effective and reliable position-tracking systems.

Keywords: Augmented Reality, AR, Microsoft HoloLens, object tracking, industrial processes.

References:

[1] Alkhimova, S., & Davydovych, I. (2022). Accuracy assessment of marker recognition using ultra wide-angle camera. Technology Audit and Production Reserves, 3(2(65)), 6–10. https://doi.org/10.15587/2706-5448.2022.259068
[2] Evans, G., Miller, J., Iglesias Pena, M., MacAllister, A., & Winer, E. (2017). Evaluating the Microsoft HoloLens through an augmented reality assembly application. Degraded Environments: Sensing, Processing, and Display 2017, 10197, 101970V. https://doi.org/10.1117/12.2262626
[3] Liu, Y., Dong, H., Zhang, L., & el Saddik, A. (2018). Technical Evaluation of HoloLens for Multimedia: A First Look. IEEE MultiMedia, 25(4), 8–18. https://doi.org/10.1109/MMUL.2018.2873473
[4] Jin, M., Gao, Y., Zeng, Y., & Zhang, J. (2018). Augmented reality-based assembly guidance for the improvement of assembly accuracy and efficiency. Robotics and Computer-Integrated Manufacturing, 51, 255-264.
[5] Büttner, S., Prilla, M., & Röcker, C. (2020). Augmented reality training for industrial assembly work - are projection-based AR assistive systems an appropriate tool for assembly training? Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3313831.3376720
[6] 3d interactive augmented reality in early stages of product design, Proceedings of 10th Conference on Human-Computer Interaction, HCI International 2003 (2003), pp. 1203-1207
[7] Etonam, A. K., Gravio, G., Kuloba, P., & Njiri, J. (2019). Augmented reality (AR) application in manufacturing encompassing quality control and maintenance. International Journal of Engineering and Advanced Technology, 9(1), 197–204. https://doi.org/10.35940/ijeat.a1120.109119
[8] Hübner, P., Clintworth, K., Liu, Q., Weinmann, M., & Wursthorn, S. (2020). Evaluation of hololens tracking and depth sensing for indoor mapping applications. Sensors (Switzerland), 20(4). https://doi.org/10.3390/s20041021
[9] Liccardo, A., & Bonavolontà, F. (2022). VR, AR, and 3-D User Interfaces for measurement and Control. Future Internet, 15(1), 18. https://doi.org/10.3390/fi15010018
[10] Kress, B. C., & Cummings, W. J. (2017). Optical architecture of HoloLens mixed reality headset. SPIE Proceedings. https://doi.org/10.1117/12.2270017
[11] Park, S., Bokijonov, S., & Choi, Y. (2021). Review of Microsoft HoloLens applications over the past five years. In Applied Sciences (Switzerland) (Vol. 11, Issue 16). MDPI AG. https://doi.org/10.3390/app11167259
[12] Farasin, A., Peciarolo, F., Grangetto, M., Gianaria, E., & Garza, P. (2020). Real-time object detection and tracking in mixed reality using Microsoft HoloLens. Proceedings of the 15th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications. https://doi.org/10.5220/0008877901650172
[13] Microsoft. (2021). HoloLens development edition. Retrieved from https://www.microsoft.com/en-us/hololens/developers
[14] Hossain, & Lee. (2019). Deep learning-based real-time multiple-object detection and tracking from aerial imagery via a flying robot with GPU-based embedded devices. Sensors, 19(15), 3371. https://doi.org/10.3390/s19153371
[15] Kunz, Christian, Maurer, Paulina, Kees, Fabian, Henrich, Pit, Marzi, Christian, Hlaváč, Michal, Schneider, Max and Mathis-Ullrich, Franziska. "Infrared marker tracking with the HoloLens for neurosurgical interventions" Current Directions in Biomedical Engineering, vol. 6, no. 1, 2020, pp. 20200027. https://doi.org/10.1515/cdbme-2020-0027
[16] PTC. (2018). HoloLens in Manufacturing: A Game Changer. PTC. https://www.ptc.com/en/products/augmented-reality/hololens-in-manufacturing
[17] Guney, C. (2017). Rethinking indoor localization solutions towards the future of mobile location-based services. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, IV-4/W4, 235–247. https://doi.org/10.5194/isprs-annals-iv-4-w4-235-2017
[18] Park, Y., Dang, L. M., Lee, S., Han, D., & Moon, H. (2021). Multiple object tracking in deep learning approaches: A survey. Electronics, 10(19), 2406. https://doi.org/10.3390/electronics10192406
[19] Feigl, T., Porada, A., Steiner, S., Löffler, C., Mutschler, C., & Philippsen, M. (2020). Localization limitations of ARCore, ARKit, and hololens in dynamic large-scale industry environments. VISIGRAPP 2020 - Proceedings of the 15th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications, 1, 307–318. https://doi.org/10.5220/0008989903070318
[20] Vassallo, R., Rankin, A., Chen, E. C., & Peters, T. M. (2017). Hologram Stability Evaluation for Microsoft hololens. Medical Imaging 2017: Image Perception, Observer Performance, and Technology Assessment. https://doi.org/10.1117/12.2255831
[21] Zhang, L., Dong, H., & El Saddik, A. (2019). Towards a qoe model to evaluate holographic augmented reality devices. IEEE MultiMedia, 26(2), 21–32. https://doi.org/10.1109/mmul.2018.2873843
[22] Kirks, T., Jost, J., Uhlott, T., Puth, J., & Jakobs, M. (2019). Evaluation of the application of smart glasses for decentralized control systems in Logistics. 2019 IEEE Intelligent Transportation Systems Conference (ITSC). https://doi.org/10.1109/itsc.2019.8917159
[23] Khoshelham, K., Tran, H., & Acharya, D. (2019). Indoor mapping eyewear: Geometric Evaluation of Spatial Mapping Capability of HoloLens. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, XLII-2/W13, 805–810. https://doi.org/10.5194/isprs-archives-xlii-2-w13-805-2019
[24] Radkowski, R., & Kanunganti, S. (2018). Augmented reality system calibration for assembly support with the Microsoft HoloLens. Volume 3: Manufacturing Equipment and Systems. https://doi.org/10.1115/msec2018-6660
[25] KR 6 R900 sixx - Kuka. (n.d.). https://www.kuka.com/-/media/kuka-downloads/imported/8350ff3ca11642998dbdc81dcc2ed44c/0000205456_en.pdf
[26] Turner, A., Coulter, D., Arya, H., & Tieto, V. (n.d.). Coordinate systems - mixed reality. Mixed Reality | Microsoft Learn. https://learn.microsoft.com/en-us/windows/mixed-reality/design/coordinate-systems#Handling_tracking_errors
[27] Spatial mapping - mixed reality. Mixed Reality | Microsoft Learn. (n.d.). https://learn.microsoft.com/en-us/windows/mixed-reality/design/spatial-mapping
[28] Egger, J., & Masood, T. (2020). Augmented reality in support of intelligent manufacturing – A systematic literature review. Computers & Industrial Engineering, 140, 106195. https://doi.org/10.1016/j.cie.2019.106195.
[29] Schumann, M., Fuchs, C., Kollatsch, C., & Klimant, P. (2021). Evaluation of augmented reality-supported approaches for product design and production processes. Procedia CIRP, 97, 160-165. https://doi.org/10.1016/j.procir.2020.05.219.
[30] Eswaran, M., & Bahubalendruni, M. V. A. R. (2022). Challenges and opportunities on AR/VR technologies for manufacturing systems in the context of industry 4.0: A state of the art review. Journal of Manufacturing Systems, 65, 260-278. https://doi.org/10.1016/j.jmsy.2022.09.016
[31] Devagiri JS, Paheding S, Niyaz Q, Yang X, Smith S. Augmented Reality and Artificial Intelligence in industry: Trends, tools, and future challenges. Expert Systems with Applications. 2022; 207:118002. https://doi.org/10.1016/j.eswa.2022.118002.