Elicitation and evaluation of zoom gestures for touchless interaction with desktop displays

ABSTRACT

Touchless gestural interaction has been widely studied and adopted in many contexts. Furthermore, the growing availability of low-cost enabling devices, such as Kinect or Leap Motion, boosted up the interest in such interaction both for commercial and scientific purposes, both for large public displays and for personal displays. The problem of choosing the right touchless gesture for the right action is thus still an open issue, depending on several aspects, such as context, purpose, users' culture, etc. In this work, we first present the results of a gesture elicitation study that allowed us to identify a set of touchless gestures for performing zoom actions while interacting with desktop displays. Next, we present a second user study for evaluating perceived workload, usability and effectiveness of the elicited gestures, showing a clear preference towards one.

References

1. Panagiotis Apostolellis, Brennon Bortz, Mi Peng, Nicholas Polys, and Andy Hoegh. 2014. Poster: Exploring the integrality and separability of the Leap Motion Controller for direct manipulation 3D interaction. In 2014 IEEE Symposium on 3D User Interfaces (3DUI). 153--154.
2. Carmelo Ardito, Paolo Buono, Maria Francesca Costabile, and Giuseppe Desolda. 2015. Interaction with Large Displays: A Survey. ACM Comput. Surv. 47, 3, Article 46 (Feb. 2015), 38 pages.
3. Moniruzzaman Bhuiyan and Rich Picking. 2011. A Gesture Controlled User Interface for Inclusive Design and Evaluative Study of Its Usability. Journal of Software Engineering and Applications 4, 9 (2011), 513.
4. Nicola Bizzotto, Alessandro Costanzo, Leonardo Bizzotto, Dario Regis, Andrea Sandri, and Bruno Magnan. 2014. Leap Motion Gesture Control With OsiriX in the Operating Room to Control Imaging: First Experiences During Live Surgery. Surgical innovation 21, 6 (December 2014), 655fi!?656.
5. Frédréric Bourgeois and Yves Guiard. 2002. Multiscale Pointing: Facilitating Pan-zoom Coordination. In CHI '02 Extended Abstracts on Human Factors in Computing Systems (CHI EA '02). ACM, New York, NY, USA, 758--759.
6. John Brooke. 1996. SUS: A 'quick and dirty' usability scale. In Usability Evaluation in Industry, P. W. Jordan, B. Weerdmeester, A. Thomas, and I. L. Mclelland (Eds.). Taylor and Francis, London.
7. Ziyun Cai, Jungong Han, Li Liu, and Ling Shao. 2017. RGB-D datasets using microsoft kinect or similar sensors: a survey. Multimedia Tools and Applications 76, 3 (01 Feb 2017), 4313--4355.
8. Andy Cockburn, Amy Karlson, and Benjamin B. Bederson. 2008. A review of overview+detail, zooming, and focus+context interfaces. Comput. Surveys 41, 1 (dec 2008), 1--31.
9. Jane L. E, Ilene L. E, James A. Landay, and Jessica R. Cauchard. 2017. Drone & Wo: Cultural Influences on Human-Drone Interaction Techniques. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17). ACM, New York, NY, USA, 6794--6799.
10. Lee Garber. 2013. Gestural Technology: Moving Interfaces in a New Direction. Computer 46, 10 (October 2013), 22--25.
11. Vito Gentile, Alessio Malizia, Salvatore Sorce, and Antonio Gentile. 2015. Designing Touchless Gestural Interactions for Public Displays In-the-Wild. Springer International Publishing, 24--34.
12. Vito Gentile, Salvatore Sorce, Alessio Malizia, Fabrizio Milazzo, and Antonio Gentile. 2017. Investigating how User Avatar in Touchless Interfaces Affects Perceived Cognitive Load and Two-Handed Interactions. In Proceedings of The 6th ACM International Symposium on Pervasive Displays (PerDis 2017). ACM.
13. Vito Gentile, Salvatore Sorce, Alessio Malizia, Dario Pirrello, and Antonio Gentile. 2016. Touchless Interfaces For Public Displays: Can We Deliver Interface Designers From Introducing Artificial Push Button Gestures?. In Proceedings of the International Working Conference on Advanced Visual Interfaces (AVI '16). ACM, New York, NY, USA, 40--43.
14. Sukeshini A. Grandhi, Gina Joue, and Irene Mittelberg. 2011. Understanding Naturalness and Intuitiveness in Gesture Production: Insights for Touchless Gestural Interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '11). ACM, New York, NY, USA, 821--824.
15. Yves Guiard and Michel Beaudouin-Lafon. 2004. Target acquisition in multiscale electronic worlds. International Journal of Human-Computer Studies 61, 6 (2004), 875 -- 905. Fitts' law 50 years later: applications and contributions from human-computer interaction.
16. Sandra G. Hart. 2006. Nasa-Task Load Index (NASA-TLX); 20 Years Later. Proceedings of the Human Factors and Ergonomics Society Annual Meeting 50, 9 (2006), 904--908.
17. Sandra G. Hart and Lowell E. Staveland. 1988. Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research. Advances in Psychology 52 (1988), 139 -- 183.
18. Ken Hinckley, Randy Pausch, John C. Goble, and Neal F. Kassell. 1994. A Survey of Design Issues in Spatial Input. In Proceedings of the 7th Annual ACM Symposium on User Interface Software and Technology (UIST '94). ACM, New York, NY, USA, 213--222.
19. Anne Köpsel and Nikola Bubalo. 2015. Benefiting from legacy bias. interactions 22, 5 (aug 2015), 44--47.
20. Panayiotis Koutsabasis and Chris K. Domouzis. 2016. Mid-Air Browsing and Selection in Image Collections. In AVI '16 Proceedings of the International Working Conference on Advanced Visual Interfaces. 21--27.
21. Arun Kulshreshth and Joseph J. LaViola, Jr. 2014. Exploring the Usefulness of Finger-based 3D Gesture Menu Selection. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '14). ACM, New York, NY, USA, 1093--1102.
22. Joseph J. LaViola and Daniel F. Keefe. 2011. 3D spatial interaction: applications for art, design, and science. In ACM SIGGRAPH 2011 Courses on - SIGGRAPH '11. ACM Press, New York, New York, USA, 1--75.
23. Jonathan Lazar, Jinjuan Heidi Feng, and Harry Hochheiser. 2010. Research Methods in Human-Computer Interaction. Wiley Publishing.
24. Alessio Malizia and Andrea Bellucci. 2012. The Artificiality of Natural User Interfaces. Commun. ACM 55, 3 (March 2012), 36--38.
25. Giulio Marin, Fabio Dominio, and Pietro Zanuttigh. 2016. Hand gesture recognition with jointly calibrated Leap Motion and depth sensor. Multimedia Tools and Applications 75, 22 (01 Nov 2016), 14991--15015.
26. Microsoft. 2014. Human Interface Guidelines 2.0. Technical Report. htp://download.microsoft.com/download/6/7/6/676611B4-1982-47A4-A42E-4CF84E1095A8/KinectHIG.2.0.pdf
27. Fabrizio Milazzo, Vito Gentile, Antonio Gentile, and Salvatore Sorce. 2018. KIND-DAMA: A modular middleware for Kinect-like device data management. Software: Practice and Experience 48, 1 (2018), 141--160.
28. MobileVenue.com. 2011. Gesture and Motion Controls on Samsung Galaxy S2 Demoed. htp://www.mobilevenue.com/gesture-motion-controls-samsung-galaxy-s2-demoed-03295125/. Accessed: 2018-05-08.
29. Meredith Ringel Morris, Andreea Danielescu, Steven Drucker, Danyel Fisher, Bongshin Lee, C. Schraefel, and Jacob O. Wobbrock. 2014. Reducing legacy bias in gesture elicitation studies. interactions 21, 3 (may 2014), 40--45.
30. Meredith Ringel Morris, Jacob O. Wobbrock, and Andrew D. Wilson. 2010. Understanding users' preferences for surface gestures. In Proceedings of Graphics Interface 2010. Canadian Information Processing Society, 261--268. http://dl.acm.org/citation.cfm?id=1839214.1839260
31. Mathieu Nancel, Julie Wagner, Emmanuel Pietriga, Olivier Chapuis, and Wendy Mackay. 2011. Mid-air pan-and-zoom on wall-sized displays. In Proceedings of the 2011 annual conference on Human factors in computing systems - CHI '11. ACM Press, New York, New York, USA, 177.
32. Jaime Ruiz, Yang Li, and Edward Lank. 2011. User-defined Motion Gestures for Mobile Interaction. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '11). ACM, New York, NY, USA, 197--206.
33. Matthias Schwaller, Simon Brunner, and Denis Lalanne. 2013. Two Handed Mid-Air Gestural HCI: Point + Command. Springer Berlin Heidelberg, Berlin, Heidelberg, 388--397.
34. Radu-Daniel Vatavu and Ionut-Alexandru Zaiti. 2014. Leap Gestures for TV: Insights from an Elicitation Study. In Proceedings of the ACM International Conference on Interactive Experiences for TV and Online Video (TVX '14). ACM, New York, NY, USA, 131--138.
35. Jacob O. Wobbrock, Htet Htet Aung, Brandon Rothrock, and Brad A. Myers. 2005. Maximizing the Guessability of Symbolic Input. In CHI '05 Extended Abstracts on Human Factors in Computing Systems (CHI EA '05). ACM, New York, NY, USA, 1869--1872.

Index Terms

• Human-centered computing

  • Human computer interaction (HCI)

    • HCI design and evaluation methods

      • User studies

    • Interaction techniques

      • Gestural input