Understanding finger input above desktop devices

ABSTRACT

Using the space above desktop input devices adds a rich new input channel to desktop interaction. Input in this elevated layer has been previously used to modify the granularity of a 2D slider, navigate layers of a 3D body scan above a multitouch table and access vertically stacked menus. However, designing these interactions is challenging because the lack of haptic and direct visual feedback easily leads to input errors. For bare finger input, the user's fingers needs to reliably enter and stay inside the interactive layer, and engagement techniques such as midair clicking have to be disambiguated from leaving the layer. These issues have been addressed for interactions in which users operate other devices in midair, but there is little guidance for the design of bare finger input in this space.

In this paper, we present the results of two user studies that inform the design of finger input above desktop devices. Our studies show that 2 cm is the minimum thickness of the above-surface volume that users can reliably remain within. We found that when accessing midair layers, users do not automatically move to the same height. To address this, we introduce a technique that dynamically determines the height at which the layer is placed, depending on the velocity profile of the user's initial finger movement into midair. Finally, we propose a technique that reliably distinguishes clicking from homing movements, based on the user's hand shape. We structure the presentation of our findings using Buxton's three-state input model, adding additional states and transitions for above-surface interactions.

References

1. A. Banerjee, J. Burstyn, A. Girouard, and R. Vertegaal. Pointable: an in-air pointing technique to manipulate out-of-reach targets on tabletops. In ITS '11, 11--20. 2011.
2. W. Buxton. A three-state model of graphical input. In INTERACT '90, 449--456. 1990.
3. F. Camp, A. Schick, and R. Stiefelhagen. How to click in mid-air. In Distributed, Ambient, and Pervasive Interactions, volume 8028 of LCNS, 78--86. 2013.
4. M. Desmurget, M. Jordan, C. Prablanc, and M. Jeannerod. Constrained and unconstrained movements involve different control strategies. J NEUROPHYSIOL, 77(3):1644--1650, 1997.
5. O. Hilliges, S. Izadi, A. D. Wilson, S. Hodges, A. Garcia-Mendoza, and A. Butz. Interactions in the air: adding further depth to interactive tabletops. In UIST '09, 139--148. 2009.
6. M. Jeannerod. The timing of natural prehension movements. J MOTOR BEHAV, 16(3):235, 1984.
7. H. Kato and H. Yanagihara. Pacman ui: vision-based finger detection for positioning and clicking manipulations. In MobileHCI '13, 464--467. 2013.
8. R. S. Kattinakere, T. Grossman, and S. Subramanian. Modeling steering within above-the-surface interaction layers. In CHI '07, 317--326. 2007.
9. J. Lee, A. Olwal, H. Ishii, and C. Boulanger. SpaceTop: integrating 2D and spatial 3D interactions in a see-through desktop environment. In CHI '13, 189--192. 2013.
10. N. Marquardt, R. Jota, S. Greenberg, and J. Jorge. The continuous interaction space: Interaction techniques unifying touch and gesture on and above a digital surface. In INTERACT '11, 461--476. 2011.
11. C. Mueller-Tomfelde, A. Wessels, and C. Schremmer. Tilted tabletops: In between horizontal and vertical workspaces. In Wksp on Horizontal Interactive Human Computer Systems, TABLETOP '08, 49--56. 2008.
12. T. Mysliwiec. Fingermouse: A freehand computer pointing interface. In Int'l Conference on Automatic Face and Gesture Recognition, 372--377. 1994.
13. M. Ortega and L. Nigay. Airmouse: Finger gesture for 2d and 3d interaction. In INTERACT '09, 214--227. 2009.
14. D. Pyryeskin, M. Hancock, and J. Hoey. Comparing Elicited Gestures to Designer-created Gestures for Selection Above a Multitouch Surface. In ITS '12, 1--10. 2012.
15. M. Spindler, M. Martsch, and R. Dachselt. Going beyond the surface: studying multi-layer interaction above the tabletop. In CHI '12, 1277--1286. 2012.
16. M. Spindler, S. Stellmach, and R. Dachselt. Paperlens: advanced magic lens interaction above the tabletop. In ITS '09, 69--76. 2009.
17. S. Subramanian, D. Aliakseyeu, and A. Lucero. Multi-layer interaction for digital tables. In UIST '06, 269--272. 2006.
18. D. Vogel and R. Balakrishnan. Distant freehand pointing and clicking on very large, high resolution displays. In UIST '05, 33--42. 2005.
19. A. D. Wilson. Robust computer vision-based detection of pinching for one and two-handed gesture input. In UIST '06, 255--258. 2006.
20. A. D. Wilson and H. Benko. Combining multiple depth cameras and projectors for interactions on, above and between surfaces. In UIST '10, 273--282. 2010.
21. J. O. Wobbrock, L. Findlater, D. Gergle, and J. J. Higgins. The aligned rank transform for nonparametric factorial analyses using only anova procedures. In CHI '11, 143--146. 2011.
22. C. Yu, X. Tan, Y. Shi, and Y. Shi. Air finger: enabling multi-scale navigation by finger height above the surface. In UbiComp '11, 495--496. 2011.

Index Terms

• Human-centered computing

  • Human computer interaction (HCI)