Touchless gestural interaction with small displays: a case study

ABSTRACT

Touchless gestural interaction enables users to interact with digital devices using body movements and gestures, and without the burden of a physical contact with technology (e.g., data gloves, body markers, or remote controllers). Most gesture-based touchless applications are designed for interaction with medium or, more often, large displays. Our research instead explores touchless gestural interaction "in-the- small", where the user interacts with small displays, of the size, for example, of a smart phone screen. Our work applies this to the domain of household appliances. We describe the design and evaluation of MOTIC (MOtion-based Touchless Interactive Cooking system), and highlight the complexity of employing touchless gestures to interact with visual interfaces constrained in size. This case study shows how existing design guidelines, mostly conceived for touchless gestural interaction "in the large", can be adapted and applied for gestural interaction in-the-small, and high-lights that touchless gestures have the potential of forming a valuable addition to our repertoire of interaction techniques with small displays.

References

1. Ali Bigdelou, Loren Arthur Schwarz, N. N. An adaptive solution for intra-operative gesture-based human-machine interaction. Proc. IUI 2012, ACM (2012), 75--84
2. Ashbrook, D., Starner, T. Magic: a motion gesture design tool. Proc. CHI 2010, ACM (2010), 2159--2168.
3. Baudel, T., and Beaudouin-Lafon, M. Charade: remote control of objects using free-hand gestures. CACM 36, 7 (1993), 28--35.
4. Bellucci, A., Malizia, A., Diaz, P., Aedo, I. Humandisplay interaction technology: Emerging remote interfaces for pervasive display environments. Pervasive Computing, IEEE 9, 2 (2010), 72--76.
5. Benavidez, P., and Jamshidi, M. Mobile robot navigation and target tracking system. Proc. SoSE 2011, (2011), 299--304.
6. Beyer, G., Alt, F., Müller, J., Schmidt, A., Isakovic, K., Klose, S., Schiewe, M., and Haulsen, I. Audience behavior around large interactive cylindrical screens. Proc. CHI 2011, ACM (2011), 1021--1030.
7. Cassell, J. A framework for gesture generation and interpretation. Computer Vision in Human-Machine Interaction (1998), 191--215.
8. De la Barré, R., Chojecki, P., Leiner, U., Mühlbach, L., and Ruschin, D. Touchless interaction-novel chances and challenges. In Human-Computer Interaction. Novel Interaction Methods and Techniques, J. Jacko, (ed.), LNCS 5611Springer (2009) 161--169.
9. Döring, T., Kern, D., Marshall, P., Pfeiffer, M., Schöning, S., Gruhn, V, Schmidt, A. Gestural interaction on the steering wheel: reducing the visual demand. Proc. CHI 2011, ACM (2011), 483--492
10. Grandhi, S. A., Joue, G., and Mittelberg, I. Understanding naturalness and intuitiveness in gesture production: insights for touchless gestural interfaces. Proc. CHI 2011, ACM (2011), 821--824.
11. http://www.microsoft.com/enus/kinectforwindows/
12. http://www.xbox.com/en us/kinect
13. Johnson, R., O'Hara, K., Sellen, A., Cousins, C., and Criminisi, A. Exploring the potential for touchless interaction in image-guided interventional radiology. Proc. CHI 2011, ACM (2011), 3323--3332.
14. Heydekorn, J., Frisch, M., Dachselt, R.: Prospects of user elicited gestural interaction techniques. CHI 2010 Workshop on Natural User Interfaces: The Prospect and Challenge of Touch and Gestural Computing..
15. Kuikkaniemi, K., Jacucci, G., Turpeinen, M., Hoggan, E., and Muandller, J. From space to stage: How interactive screens will change urban life. Computer 44, 6 (2011), 40--47.
16. Kurtenbach, G., and Hulteen., E. Gestures in Human-Computer Communication. Addison-Wesley(1990).
17. Maged N., Kamel B., Bryan J., Blanchard, C. W. J. M. A. T., and Gutierrez-Osuna, R. Web gis in practice: a microsoft kinect natural user interface for google earth navigation. Int. J. health geographics (2011), 10--45.
18. Malizia, A. and Bellucci, A. The artificiality of natural user interfaces. CACM 55, 3 (March 2012), 36--38.
19. Ng, J., Sim, T.-J., Foo, Y.-S., and Yeo, V. Gesture-based interaction with virtual 3d objects on large display: what makes it fun? Proc. CHI EA 2009, ACM (2009), 3751--3756.
20. Nielsen, M., et al. Moeslund, T., Storring, M. and Granum, E. A Procedure for developing intuitive and ergonomic gesture interfaces for HCI. Proc. Gesture-Based Communication in Human-Computer Interaction: 5th Int. Gesture Workshop, GW 2003, LNCS 2915, Springer, Berlin (2004), 409--420.
21. Norman, D. A, Natural user interfaces are not natural. interactions, v.17 n.3, May + June 2010
22. Norman, D. A, Nielsen, J. Gestural interfaces: a step backward in usability. interactions, v.17 n.5, September + October 2010
23. Pickering, C. A, Burnham, K. J, Richardson, M. J. A Research Study of Hand Gesture Recognition Technologies and Applications for Human Vehicle Interaction. Proc. Automotive Electronics, 3rd Institution of Engineering and Technology Conference (2007), 1--15
24. Riener A. Gestural Interaction in Vehicular Applications. IEEE Computer, 42--47
25. Scoditti, A., Blanch, R., and Coutaz.J. A novel taxonomy for gestural interaction techniques based on accelerometers. Proc. IUI 2011, ACM (2011), 63--72
26. Shan, C. Gesture control for consumer electronics. In Multimedia Interaction and Intelligent User Interfaces, L. Shao, C. Shan, J. Luo, and M. Etoh, (eds.), Advances in Pattern Recognition. Springer, 2010, pp. 107--128.
27. Sven, G., Markus L., F. D. M. B. A. K. Using intelligent natural user interfaces to support sales conversations. Proc. IUI 2012, ACM (2012), 97--100
28. Vatavu, R.,. 2012. User-defined gestures for free-hand TV control. In Proceedings of the 10th European conference on Interactive tv and video (EuroiTV '12). ACM, 45--48.
29. Wachs, J. P., Kölsch, M., Stern, H., and Edan, Y. Vision-based hand-gesture applications. CACM 54, 2 (2011), 60--71.
30. Wei, J., Wang, X., Peiris, R. L., Choi, Y., Martinez, X. R., Tache, R., Koh, J. T. K. V., Halupka, V., and Cheok, A. D. Codine: an interactive multi-sensory system for remote dining. Proc. UbiComp 2011, ACM (2011), 21--30.
31. Wobbrock, J. O., Morris, M. R., and Wilson, A. D. 2009. User-defined gestures for surface computing. In Proc. CHI '09. ACM (2009), 1083--1092.

Index Terms

• Human-centered computing

  • Human computer interaction (HCI)