In-Situ Instructions Exceed Side-by-Side Instructions in Augmented Reality Assisted Assembly

ABSTRACT

Driven by endeavors towards Industry 4.0, there is increasing interest in augmented reality (AR) as an approach for assistance in areas like picking, assembly and maintenance. In this work our focus is on AR-based assistance in manual assembly. The design space for AR instructions in this context includes, e.g., side-by-side, 3D or projected 2D presentations. In previous research, the low quality of the AR devices available at the respective time had a significant impact on performance evaluations. Today, a proper and up-to-date comparison of different presentation approaches is missing.

This paper presents an improved 3D in-situ instruction and compares it to previously presented techniques. All instructions are implemented on up-to-date AR hardware, namely the Microsoft HoloLens. To support reproducible research, the comparison is made using a standardized benchmark scenario. The results show, contrary to previous research, that in-situ instructions on state-of-the-art AR glasses outperform side-by-side instructions in terms of errors made, task completion time, and perceived task load.

References

1. H. Alvarez, Iker Aguinaga, and Diego Borro. 2011. Providing guidance for maintenance operations using automatic markerless Augmented Reality system. In 2011 10th IEEE International Symposium on Mixed and Augmented Reality (ISMAR). 181--190.
2. Alexander Bannat, Frank Wallhoff, Gerhard Rigoll, Florian Friesdorf, H. Bubb, Sonja Stork, H. J. Müller, Anna Schubö, Mathey Wiesbeck, and Michael F. Zäh. 2008. Towards optimal worker assistance: a framework for adaptive selection and presentation of assembly instructions. In Proceedings of the 1st international workshop on cognition for technical systems, Cotesys. http://www.academia.edu/download/42003151/Towards_Optimal_Worker_Assistance_A_Fram20160203-32507-1ncxacj.pdf
3. B. Besbes, S.N. Collette, M. Tamaazousti, S. Bourgeois, and V. Gay-Bellile. 2012. An interactive Augmented Reality system: A prototype for industrial maintenance training applications. In 2012 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). 269--270.
4. Jonas Blattgerste, Benjamin Strenge, Patrick Renner, Thies Pfeiffer, and Kai Essig. 2017. Comparing Conventional and Augmented Reality Instructions for Manual Assembly Tasks. In Proceedings of the 10th International Conference on Pervasive Technologies Related to Assistive Environments (PETRA'17). ACM.
5. Markus Funk, Andreas Bächler, Liane Bächler, Oliver Korn, Christoph Krieger, Thomas Heidenreich, and Albrecht Schmidt. 2015. Comparing Projected Insitu Feedback at the Manual Assembly Workplace with Impaired Workers. In Proceedings of the 8th ACM International Conference on PErvasive Technologies Related to Assistive Environments (PETRA '15). ACM, New York, NY, USA, 1:1--1:8.
6. Markus Funk, Thomas Kosch, Scott W. Greenwald, and Albrecht Schmidt. 2015. A Benchmark for Interactive Augmented Reality Instructions for Assembly Tasks. In Proceedings of the 14th International Conference on Mobile and Ubiquitous Multimedia (MUM '15). ACM, New York, NY, USA, 253--257.
7. Markus Funk, Thomas Kosch, and Albrecht Schmidt. 2016. Interactive Worker Assistance: Comparing the Effects of In-situ Projection, Head-mounted Displays, Tablet, and Paper Instructions. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp '16). ACM, New York, NY, USA, 934--939.
8. D. Gorecky, M. Schmitt, M. Loskyll, and D. Zühlke. 2014. Human-machine-interaction in the industry 4.0 era. In 2014 12th IEEE International Conference on Industrial Informatics (INDIN). 289--294. 00067.
9. Anhong Guo, Shashank Raghu, Xuwen Xie, Saad Ismail, Xiaohui Luo, Joseph Simoneau, Scott Gilliland, Hannes Baumann, Caleb Southern, and Thad Starner. 2014. A Comparison of Order Picking Assisted by Head-up Display (HUD), Cart-mounted Display (CMD), Light, and Paper Pick List. In Proceedings of the 2014 ACM International Symposium on Wearable Computers (ISWC '14). ACM, New York, NY, USA, 71--78.
10. Ankit Gupta, Dieter Fox, Brian Curless, and Michael Cohen. 2012. Duplo-Track: A Real-time System for Authoring and Guiding Duplo Block Assembly. In Proceedings of the 25th Annual ACM Symposium on User Interface Software and Technology (UIST '12). ACM, New York, NY, USA, 389--402.
11. Julie Heiser, Doantam Phan, Maneesh Agrawala, Barbara Tversky, and Pat Hanrahan. 2004. Identification and Validation of Cognitive Design Principles for Automated Generation of Assembly Instructions. In Proceedings of the Working Conference on Advanced Visual Interfaces (AVI '04). ACM, New York, NY, USA, 311--319.
12. S. Henderson and S. Feiner. 2011. Exploring the Benefits of Augmented Reality Documentation for Maintenance and Repair. IEEE Transactions on Visualization and Computer Graphics 17, 10 (2011), 1355--1368.
13. S. J. Henderson and S. Feiner. 2009. Evaluating the benefits of augmented reality for task localization in maintenance of an armored personnel carrier turret. In 2009 8th IEEE International Symposium on Mixed and Augmented Reality. 135--144.
14. B. M. Khuong, K. Kiyokawa, A. Miller, J. J. La Viola, T. Mashita, and H. Takemura. 2014. The effectiveness of an AR-based context-aware assembly support system in object assembly. In 2014 IEEE Virtual Reality (VR). 57--62.
15. Oliver Korn, Markus Funk, Stephan Abele, Thomas Hörz, and Albrecht Schmidt. 2014. Context-aware Assistive Systems at the Workplace: Analyzing the Effects of Projection and Gamification., Article 38 (2014), 8 pages.
16. Oliver Korn, Markus Funk, and Albrecht Schmidt. 2015. Assistive systems for the workplace: Towards context-aware assistance. Assistive Technologies for Physical and Cognitive Disabilities (2015), 121--133.
17. Oliver Korn, Albrecht Schmidt, and Thomas Hörz. 2013. Augmented Manufacturing: A Study with Impaired Persons on Assistive Systems Using In-situ Projection. In Proceedings of the 6th International Conference on PErvasive Technologies Related to Assistive Environments (PETRA '13). ACM, New York, NY, USA, 21:1--21:8.
18. Robert E. Kraut, Susan R. Fussell, and Jane Siegel. 2003. Visual Information As a Conversational Resource in Collaborative Physical Tasks. Hum.-Comput. Interact. 18, 1 (June 2003), 13--19.
19. N. Petersen, A. Pagani, and D. Stricker. 2013. Real-time modeling and tracking manual workflows from first-person vision. In 2013 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). 117--124.
20. Rupert Reif and Willibald A. Günthner. 2009. Pick-by-vision: augmented reality supported order picking. The Visual Computer 25, 5-7 (March 2009), 461--467.
21. Leonardo Rodriguez, Fabian Quint, Dominic Gorecky, David Romero, and Héctor R. Siller. 2015. Developing a Mixed Reality Assistance System Based on Projection Mapping Technology for Manual Operations at Assembly Workstations. Procedia Computer Science 75 (Jan. 2015), 327--333.
22. Oliver Sand, Sebastian Büttner, Volker Paelke, and Carsten Röcker. 2016. smARt.Assembly - Projection-Based Augmented Reality for Supporting Assembly Workers. In Virtual, Augmented and Mixed Reality (Lecture Notes in Computer Science), Stephanie Lackey and Randall Shumaker (Eds.). Springer International Publishing, 643--652. http://link.springer.com/chapter/10.1007/978-3-319-39907-2_61
23. Angela L. Sauer, Andra Parks, and Patricia C. Heyn. 2010. Assistive technology effects on the employment outcomes for people with cognitive disabilities: A systematic review. Disability and Rehabilitation. Assistive Technology 5, 6 (2010), 377--391.
24. Bjorn Schwerdtfeger and Gudrun Klinker. 2008. Supporting Order Picking with Augmented Reality. In Proceedings of the 7th IEEE/ACM International Symposium on Mixed and Augmented Reality (ISMAR 08). IEEE Computer Society, Washington, DC, USA, 91--94.
25. D. Stanimirovic, N. Damasky, S. Webel, D. Koriath, A. Spillner, and D. Kurz. 2014. {Poster} A Mobile Augmented reality system to assist auto mechanics. In 2014 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). 305--306.
26. Arthur Tang, Charles Owen, Frank Biocca, and Weimin Mou. 2003. Comparative Effectiveness of Augmented Reality in Object Assembly. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '03). ACM, New York, NY, USA, 73--80.

Index Terms

• Human-centered computing

  • Human computer interaction (HCI)

    • Interaction paradigms

      • Mixed / augmented reality

  • Interaction design

    • Interaction design process and methods