Multi-Modal Visualization of Working Instructions for Assembly Operations

Authors: Josef Wolfartsberger, Michael Heiml, Georg Schwarz, Sabrina Egger

Abstract:

Growing individualization and higher numbers of variants in industrial assembly products raise the complexity of manufacturing processes. Technical assistance systems considering both procedural and human factors allow for an increase in product quality and a decrease in required learning times by supporting workers with precise working instructions. Due to varying needs of workers, the presentation of working instructions leads to several challenges. This paper presents an approach for a multi-modal visualization application to support assembly work of complex parts. Our approach is integrated within an interconnected assistance system network and supports the presentation of cloud-streamed textual instructions, images, videos, 3D animations and audio files along with multi-modal user interaction, customizable UI, multi-platform support (e.g. tablet-PC, TV screen, smartphone or Augmented Reality devices), automated text translation and speech synthesis. The worker benefits from more accessible and up-to-date instructions presented in an easy-to-read way.

Keywords: Assembly, assistive technologies, augmented reality, manufacturing, visualization.

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.2571930

References:

[1] S. B¨uttner, H. Mucha, M. Funk, T. Kosch, M. Aehnelt, S. Robert, and C. R¨ocker, “The design space of augmented and virtual reality applications for assistive environments in manufacturing: A visual approach,” in Proceedings of the 10th International Conference on PErvasive Technologies Related to Assistive Environments, ser. PETRA ’17. New York, NY, USA: ACM, 2017, pp. 433–440.
[2] M. Funk, T. Kosch, and A. Schmidt, “Interactive worker assistance: comparing the effects of in-situ projection, head-mounted displays, tablet, and paper instructions,” in Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing - UbiComp16. ACM Press, 2016.
[3] J. Blattgerste, B. Strenge, P. Renner, T. Pfeiffer, and K. Essig, “Comparing conventional and augmented reality instructions for manual assembly tasks,” in Proceedings of the 10th International Conference on PErvasive Technologies Related to Assistive Environments - PETRA17. ACM Press, 2017.
[4] S. B¨uttner, M. Funk, O. Sand, and C. R¨ocker, “Using head-mounted displays and in-situ projection for assistive systems: A comparison,” in Proceedings of the 9th ACM International Conference on PErvasive Technologies Related to Assistive Environments, ser. PETRA ’16. New York, NY, USA: ACM, 2016, pp. 44:1–44:8.
[5] M. Funk, T. Kosch, S. W. Greenwald, and A. Schmidt, “A benchmark for interactive augmented reality instructions for assembly tasks,” in Proceedings of the 14th International Conference on Mobile and Ubiquitous Multimedia - MUM15. ACM Press, 2015.
[6] R. Radkowski, J. Herrema, and J. Oliver, “Augmented reality-based manual assembly support with visual features for different degrees of difficulty,” International Journal of Human-Computer Interaction, vol. 31, no. 5, pp. 337–349, jan 2015.
[7] A. Tang, C. Owen, F. Biocca, and W. Mou, “Comparative effectiveness of augmented reality in object assembly,” in Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ser. CHI ’03. New York, NY, USA: ACM, 2003, pp. 73–80.
[8] M. Funk, A. Bchler, L. Bchler, O. Korn, C. Krieger, T. Heidenreich, and A. Schmidt, “Comparing projected in-situ feedback at the manual assembly workplace with impaired workers,” in Proceedings of the 8th ACM International Conference on PErvasive Technologies Related to Assistive Environments - PETRA15. ACM Press, 2015.
[9] D. Li, S. Mattsson, A˚ . Fast-Berglund, and M. A˚ kerman, “Testing operator support tools for a global production strategy,” Procedia CIRP, vol. 44, pp. 120–125, 2016.
[10] M. Sderberg, Johansson, “Development of simple guidelines to improve assembly instructions and operator performance,” in Proceedings of the sixth Swedish Production Symposium. Chalmers Research, 2014.
[11] S. Mattsson and A˚ . Fast-Berglund, “How to support intuition in complex assembly?” Procedia CIRP, vol. 50, pp. 624–628, 2016.
[12] S. Mattsson, A˚ . Fast-Berglund, and D. Li, “Evaluation of guidelines for assembly instructions,” IFAC-PapersOnLine, vol. 49, no. 12, pp. 209–214, 2016.
[13] T. Fa¨ssberg, A˚ . Fasth, S. Mattsson, and J. Stahre, “Cognitive automation in assembly systems for mass customization,” in Proceedings of the 4th Swedish Production Symposium (SPS), Lund, Sweden, 2011.
[14] M. Agrawala, D. Phan, J. Heiser, J. Haymaker, J. Klingner, P. Hanrahan, and B. Tversky, “Designing effective step-by-step assembly instructions,” ACM Transactions on Graphics, vol. 22, no. 3, p. 828, jul 2003.
[15] R. Lindorfer, R. Froschauer, and G. Schwarz, “Adapt - a decision-model-based approach for modeling collaborative assembly and manufacturing tasks,” in 2018 IEEE 16th International Conference on Industrial Informatics (INDIN), 2018, pp. 559–564.
[16] S. Hu, J. Ko, L. Weyand, H. ElMaraghy, T. Lien, Y. Koren, H. Bley, G. Chryssolouris, N. Nasr, and M. Shpitalni, “Assembly system design and operations for product variety,” CIRP Annals, vol. 60, no. 2, pp. 715–733, 2011.
[17] L. Gong, D. Li, S. Mattsson, M. A˚ kerman, and A˚ . F. Berglund, “The comparison study of different operator support tools for assembly task in the era of global production,” Procedia Manufacturing, vol. 11, pp. 1271–1278, 2017.