The design and evaluation of multitouch marking menus

ABSTRACT

Despite the considerable quantity of research directed towards multitouch technologies, a set of standardized UI components have not been developed. Menu systems provide a particular challenge, as traditional GUI menus require a level of pointing precision inappropriate for direct finger input. Marking menus are a promising alternative, but have yet to be investigated or adapted for use within multitouch systems. In this paper, we first investigate the human capabilities for performing directional chording gestures, to assess the feasibility of multitouch marking menus. Based on the positive results collected from this study, and in particular, high angular accuracy, we discuss our new multitouch marking menu design, which can increase the number of items in a menu, and eliminate a level of depth. A second experiment showed that multitouch marking menus perform significantly faster than traditional hierarchal marking menus, reducing acquisition times in both novice and expert usage modalities.

References

1. Apple iPhone, http://www.apple.com/iphone/
2. Bailly, G., Lecolinet, E., and Nigay, L. (2008). Flower menus: a new type of marking menu with large menu breadth, within groups and efficient expert mode memorization. ACM AVI. 15--22.
3. Bau, O. and Mackay, W.E. (2008). OctoPocus: a dynamic guide for learning gesture-based command sets. ACM UIST. 37--46.
4. Benko, H., Wilson, A.D., Baudisch, P. (2006). Precise selection techniques for multi-touch screens. ACM CHI. 1263--1272.
5. Brandl, P., Forlines, C. Wigdor, D., Haller, M., Shen, C. (2008). Combining and Measuring the Benefits of Bimanual Pen and Direct-Touch Interaction on Horizontal Interfaces. ACM AVI. 154--161.
6. Brandl, P., Seifried, T., Leitner, J., Haller, M., Doray, B., and To, P. (2009). Occlusion-Aware Menu Design for Digital Tabletops. ACM CHI. 3223--3228.
7. Card, S.K. (1982). User perceptual mechanisms in the search of computer command menus. ACM CHI. 190--196.
8. Davidson, P.L. and Han, J.Y. (2008). Extending 2D object arrangement with pressure-sensitive layering cues. ACM UIST. 87--90.
9. Dietz, P. and Leigh, D. (2001). DiamondTouch: a multi-user touch technology. ACM UIST. 219--226.
10. Esenther, A. and Ryall, K. (2006). Fluid DTMouse: better mouse support for touch-based interactions. AVI. 112--115.
11. Fingerworks, Inc. (2008). User's Guide. http://www.fingerworks.com/gesture_guide_mouse.html
12. Freeman, D., Benko, H., Ringel-Morris, M., Wigdor, D. (2009). ShadowGuides: Visualizations for In-Situ Learning of Multi-Touch and Whole-Hand Gestures. ACM ITS.
13. Han, J.Y. (2005). Low-cost multi-touch sensing through frustrated total internal reflection. ACM UIST. 115--118.
14. Jacob, R, A. Girouard, L.M. Hirshfield, M.S. Horn, O. Shaer, E.T. Solovey, and J. Zigelbaum. (2008). Reality-Based Interaction: A Framework for Post-WIMP Interfaces. ACM CHI. 201--210.
15. Kurtenbach, G., Buxton, W. (1993). The limits of expert performance using hierarchic marking menus. ACM Interact/CHI. 482--487.
16. Kurtenbach, G., Sellen, A., Buxton, W. (1993). An empirical evaluation of some articulatory and cognitive aspects of marking menus. Journal of Human Computer Interaction, 8(1):1--23.
17. Kurtenbach, G., Fitzmaurice, G.W., Owen, R.N., and Baudel, T. (1999). The Hotbox: efficient access to a large number of menu-items. ACM CHI. 231--237.
18. Lenman, S., Bretzner, L., and Thuresson, B. (2002). Using marking menus to develop command sets for computer vision based hand gesture interfaces. ACM NORDCHI. 239--242.
19. Li, Y., Hinckley, K., Guan, Z., and Landay, J. (2005). Experimental Analysis of Mode Switching Techniques in Pen-based User Interfaces. ACM CHI. 461--470.
20. Lyons, K., Starner, T., Gane, B. (2006). Experimental evaluations of the Twiddler one--handed chording mobile keyboard. Human-Computer Interaction. 21(4):343--392.
21. Malik, S., Ranjan, A., Balakrishnan. R. (2005). Interacting with large displays from a distance with vision-tracked multi-finger gestural input. ACM UIST. 43--52.
22. Matejka, J., Grossman, T., Lo, J., and Fitzmaurice, G. (2009). The design and evaluation of multi-finger mouse emulation techniques. ACM CHI. 1073--1082.
23. Potter, R.L., Weldon, L.J., Shneiderman, B. (1988). Improving the accuracy of touch screens: an experimental evaluation of three strategies. ACM CHI. 27--32.
24. Rekimoto, J. (2002). SmartSkin: an infrastructure for freehand manipulation on interactive surfaces. ACM CHI. 113--120.
25. Rosenberg, I.D., Grau, A., Hendee, C., Awad, N., and Perlin, K. (2009). IMPAD: an inexpensive multi-touch pressure acquisition device. ACM CHI EA, 3217--3222.
26. Scott, S., Carpendale, S., and Inkpen, K. (2004). Territoriality in collaborative tabletop workspaces. ACM CSCW. 294--303.
27. Shen, C., Vernier, F.D., Forlines, C., Ringel, M. (2004). DiamondSpin: An Extensible Toolkit for Around--the--Table Interaction. ACM CHI. 167--174.
28. Vogel, D. and Baudisch, P. (2007). Shift: a technique for operating pen-based interfaces using touch. ACM CHI. 657--666.
29. Wang, F., Ren, X., Cao, X., Irani, P. (2009). Detecting and Leveraging Finger Orientation for Interaction with Direct-Touch Surfaces. ACM UIST. 23--32.
30. Wigdor, D., Balakrishnan, R. (2004). A comparison of consecutive and concurrent input text entry techniques for mobile phones. ACM CHI. 81--88.
31. Wigdor, D., Fletcher, J., and Morrison, G. (2009). Designing user interfaces for multi-touch and gesture devices. ACM CHI. 2755--2758.
32. Wilson, A. (2005). PlayAnywhere: A Compact Tabletop Computer Vision System. ACM UIST. 83--92.
33. Wobbrock, J.O., Morris, M.R. and Wilson, A.D. (2009) User-defined gestures for surface computing. ACM CHI. 1083--1092.
34. Wu, M., Balakrishnan, R. (2003). Multi-finger and whole hand gestural interaction techniques for multi-user tabletop displays. ACM UIST. 193--202.
35. Zhao, S., Balakrishnan, R. (2004). Simple vs. compound mark hierarchical marking menus. ACM UIST. 33--42.
36. Zhao, S., Agrawala, M., Hinckley, K. (2006). Zone and polygon menus: using relative position to increase the breadth of multi-stroke marking menus. ACM CHI. 1077--1086.

Index Terms

• Human-centered computing

  • Human computer interaction (HCI)

    • Interaction paradigms