The Effects of Encumbrance and Mobility on Touch-Based Gesture Interactions for Mobile Phones

ABSTRACT

In this paper, we investigate the effects of mobility and encumbrance (holding objects such as shopping bags) on standard gestures commonly performed on touchscreens: tapping, dragging, spreading & pinching and rotating clockwise & anticlockwise when completed using a two-handed input posture. These one- and two- finger on-screen gesture inputs have become common but previous research has only examined tapping performance in everyday walking and encumbered situations. Therefore, a series of Fitts' Law style targeting tasks was designed to measure the performance of each gesture with users walking only and walking while carrying bags. The results showed that encumbrance and walking had a negative impact on each gesture in terms of accuracy except for rotational actions, which were performed well. Tapping and dragging both performed poorly which shows the input difficulties of single finger interactions when encumbered and on the move. Our findings will help designers choose the appropriate input techniques for future mobile user interfaces and apps in physically demanding contexts.

References

1. Avery, J., Choi, M., Vogel, D., and Lank, E. Pinch-to-zoom-plus. Proceedings of UIST 2014, 595--604.
2. Barnard, L., Yi, J.S., Jacko, J.A., and Sears, A. An empirical comparison of use-in-motion evaluation scenarios for mobile computing devices. International Journal of Human Computer Studies 62, 4 (2005), 487--520.
3. Bergstrom-Lehtovirta, J., Oulasvirta, A., and Brewster, S. The effects of walking speed on target acquisition on a touchscreen interface. Proceedings of MobileHCI 2011, 143--146.
4. Cockburn, A., Ahlström, D., and Gutwin, C. Understanding performance in touch selections: Tap, drag and radial pointing drag with finger, stylus and mouse. International Journal of Human-Computer Studies 70, 3 (2012), 218--233.
5. Crossan, A., Murray-Smith, R., Brewster, S., Kelly, J., and Musizza, B. Gait Phase Effects in Mobile Interaction. Extended Abstracts on Human Factors in Computing Systems - CHI EA 2005, 1312--1315.
6. Findlater, L., Froehlich, J.E., Fattal, K., Wobbrock, J.O., and Dastyar, T. Age-related differences in performance with touchscreens compared to traditional mouse input. Proceedings of CHI 2013, 343--346.
7. Fitts, P.M. The information capacity of the human motor system in controlling the amplitude of movement. 1954. Journal of experimental psychology. General 121, 6 (1992), 262--269.
8. Goel, M., Findlater, L., and Wobbrock, J. WalkType: Using Accelerometer Data to Accommodate Situational Impairments in Mobile Touch Screen Text Entry. Proceedings of CHI 2012, 2687--2696.
9. Hoggan, E., Williamson, J., Oulasvirta, A., Nacenta, M., Kristensson, P.O., and Lehtiö, A. Multi-touch rotation gestures: performance and ergonomics. Proceedings of CHI 2013, 3047--3050.
10. Hoggan, E., Williamson, J., Oulasvirta, A., Nacenta, M., Kristensson, P.O., and Lehtiö, A. Multi-touch pinch gestures: performance and ergonomics. Proceedings of ITS 2013, 219--222.
11. Hoober, S. How Do Users Really Hold Mobile Devices? www.uxmatters.com
12. Kane, S.K., Wobbrock, J.O., and Smith, I.E. Getting off the treadmill: evaluating walking user interfaces for mobile devices in public spaces. Proceedings of MobileHCI 2008, 109--118.
13. Karlson, A., Bederson, B., and Contreras-Vidal, J. Understanding single-handed mobile device interaction. Technical Report 2006.
14. Kobayashi, M., Hiyama, A., Miura, T., Asakawa, C., Hirose, M., and Ifukube, T. Elderly user evaluation of mobile touchscreen interactions. Proceeding of INTERACT 2011, 83--99.
15. Lin, M., Goldman, R., Price, K.J., Sears, A., and Jacko, J. How do people tap when walking? An empirical investigation of nomadic data entry. International Journal of Human-Computer Studies 65, 9 (2007), 759--769.
16. MacKenzie, I.S. and Isokoski, P. Fitts' throughput and the speed-accuracy tradeoff. Proceedings of CHI 2008, 1633--1636.
17. MacKenzie, I.S., Kauppinen, T., and Silfverberg, M. Accuracy measures for evaluating computer pointing devices. Proceedings of CHI 2001, 9--16.
18. MacKenzie, I.S. Fitts' Law as a Research and Design Tool in Human-Computer Interaction. Human-Computer Interaction 7, 1 (1992), 91--139.
19. Mainwaring, S.D., Anderson, K., and Chang, M.F. Living for the global city: mobile kits, urban interfaces, and ubicomp. Proceeding of UbiComp 2005, 269--286.
20. Mizobuchi, S., Chignell, M., and Newton, D. Mobile text entry: relationship between walking speed and text input task difficulty. Proceedings of MobileHCI 2005, 122--128.
21. Ng, A., Brewster, S., and Williamson, J. The Impact of Encumbrance on Mobile Interactions. Proceeding of INTERACT 2013, 92--109.
22. Ng, A., Brewster, S.A., and Williamson, J.H. Investigating the effects of encumbrance on one- and two- handed interactions with mobile devices. Proceedings of CHI 2014, 1981--1990.
23. Ng, A., Williamson, J.H., and Brewster, S.A. Comparing evaluation methods for encumbrance and walking on interaction with touchscreen mobile devices. Proceedings of MobileHCI 2014, 23--32.
24. Nicolau, H. and Jorge, J. Touch typing using thumbs: understanding the effect of mobility and hand posture. Proceedings of CHI 2012, 2683--2686.
25. Oulasvirta, A. and Bergstrom-Lehtovirta, J. Ease of juggling: studying the effects of manual multitasking. Proceedings of CHI 2011, 3103--3112.
26. Schedlbauer, M. and Heines, J. Selecting While Walking : An Investigation of Aiming Performance in a Mobile Work Context Selecting While Walking : An Investigation of Aiming Performance in a Mobile Work Context. Proceedings of AMCIS 2007.
27. Schildbach, B. and Rukzio, E. Investigating Selection and Reading Performance on a Mobile Phone while Walking. Proceedings of MobileHCI 2010, 93--102.
28. Sears, A., Lin, M., Jacko, J., and Yang, X. When computers fade: Pervasive computing and situationally induced impairments and disabilities. Proceedings of HCI Int'l 2003, 1298--1302.
29. Tran, J.J., Trewin, S., Swart, C., John, B.E., and Thomas, J.C. Exploring pinch and spread gestures on mobile devices. Proceedings of MobileHCI 2013, 151--160.
30. Wobbrock, J.O., Shinohara, K., and Jansen, A. The effects of task dimensionality, endpoint deviation, throughput calculation, and experiment design on pointing measures and models. Proceedings of CHI 2011, 1639--1648.
31. Zhao, J., Soukoreff, R.W., and Balakrishnan, R.A Model of Multi-touch Manipulation. Proceedings of GRAND 2011.

Index Terms

• Human-centered computing

  • Human computer interaction (HCI)