Exploring Accessible Smartwatch Interactions for People with Upper Body Motor Impairments

ABSTRACT

Smartwatches are always-available, provide quick access to information in a mobile setting, and can collect continuous health and fitness data. However, the small interaction space of these wearables may pose challenges for people with upper body motor impairments. To investigate accessible smartwatch interactions for this user group, we conducted two studies. First, we assessed the accessibility of existing smartwatch gestures with 10 participants with motor impairments. We found that not all participants were able to complete button, swipe and tap interactions. In a second study, we adopted a participatory approach to explore smartwatch gesture preferences and to gain insight into alternative, more accessible smartwatch interaction techniques. Eleven participants with motor impairments created gestures for 16 common smartwatch actions on both touchscreen and non-touchscreen (bezel, wristband) areas of the watch and the user's body. We present results from both studies and provide design recommendations.

References

1. Youngseok Ahn, Sungjae Hwang, HyunGook Yoon, Junghyeon Gim, and Jung-hee Ryu. 2015. BandSense: Pressure-sensitive Multi-touch Interaction on a Wristband. Proc. CHI EA '15, 251--254.
2. Lisa Anthony, YooJin Kim, and Leah Findlater. 2013. Analyzing user-generated youtube videos to understand touchscreen use by people with motor impairments. Proc. CHI '13, 1223--1232.
3. Apple. Human Interface Guidelines. Retrieved September 19, 2017 from https://developer.apple.com/watchos/human-interface-guidelines/interface-elements/buttons/
4. Shaikh Shawon Arefin Shimon, Courtney Lutton, Zichun Xu, Sarah Morrison-Smith, Christina Boucher, and Jaime Ruiz. 2016. Exploring Non-touchscreen Gestures for Smartwatches. Proc. CHI '16, 3822--3833.
5. Daniel L. Ashbrook, James R. Clawson, Kent Lyons, Thad E. Starner, and Nirmal Patel. 2008. Quickdraw: the impact of mobility and on-body placement on device access time. Proc. CHI '08, 219--222.
6. Virginia Braun and Victoria Clarke. 2006. Using thematic analysis in psychology. Qualitative Research in Psychology 3, 2: 77--101.
7. Patrick Carrington, Amy Hurst, and Shaun K. Kane. 2014. Wearables and chairables: inclusive design of mobile input and output techniques for power wheelchair users. Proc. CHI '14, 3103--3112.
8. Patrick Carrington, Amy Hurst, and Shaun K. Kane. 2014. The Gest-Rest?: A Pressure-Sensitive Chairable Input Pad for Power Wheelchair Armrests. Proc. ASSETS '14, 201--208.
9. Victor Dibia. 2016. FOQUS: A Smartwatch Application for Individuals with ADHD and Mental Health Challenges. Proc. ASSETS '16, 311--312.
10. Victor Dibia, Shari Trewin, Maryam Ashoori, et al. 2015. Exploring the Potential of Wearables to Support Employment for People with Mild Cognitive Impairment. Proc. ASSETS '15, 401--402.
11. Tanja Döring, Dagmar Kern, Paul Marshall, et al. 2011. Gestural interaction on the steering wheel: reducing the visual demand. Proc. CHI '11, 483--492.
12. S. N. Duff, C. B. Irwin, J. L. Skye, M. E. Sesto, et al. 2010. The Effect of Disability and Approach on Touch Screen Performance during a Number Entry Task. Proceedings of the Human Factors and Ergonomics Society Annual Meeting 54, 6: 566--570.
13. Leah Findlater, Karyn Moffatt, Jon Froehlich, Meethu Malu, and Joan Zhang. 2017. Comparing Touchscreen and Mouse Input Performance by People With and Without Upper Body Motor Impairments. Proc. CHI '17, 6056--6061.
14. Jon Froehlich, Jacob O. Wobbrock, and Shaun K. Kane. 2007. Barrier pointing: using physical edges to assist target acquisition on mobile device touch screens. Proc. ASSETS '07, 19--26.
15. Markus Funk, Alireza Sahami, Niels Henze, and Albrecht Schmidt. 2014. Using a touch-sensitive wristband for text entry on smart watches. Proc. CHI EA '14, 2305--2310.
16. Jun Gong, Xing-Dong Yang, and Pourang Irani. 2016. WristWhirl: One-handed Continuous Smartwatch Input using Wrist Gestures. Proc.UIST '16, 861--872.
17. Tiago João Vieira Guerreiro, Hugo Nicolau, Joaquim Jorge, et al. 2010. Assessing mobile touch interfaces for tetraplegics. Proc. MobileHCI '10, 31--34.
18. Anhong Guo and Tim Paek. 2016. Exploring tilt for no-touch, wrist-only interactions on smartwatches. Proc. MobileHCI '16, 17--28.
19. Chris Harrison and Scott E. Hudson. 2009. Abracadabra: wireless, high-precision, and unpowered finger input for very small mobile devices. Proc. UIST '09, 121--124.
20. Xueliang Huo, Hangue Park, and Maysam Ghovanloo. 2012. Dual-mode tongue drive system: using speech and tongue motion to improve computer access for people with disabilities. Proc. WH '12, 1--8.
21. Curt B. Irwin and Mary E. Sesto. 2012. Performance and touch characteristics of disabled and non-disabled participants during a reciprocal tapping task using touch screen technology. Applied Ergonomics 43, 6: 1038--1043.
22. Shaun K. Kane, Chandrika Jayant, Jacob O. Wobbrock, and Richard E. Ladner. 2009. Freedom to roam: a study of mobile device adoption and accessibility for people with visual and motor disabilities. Proc. ASSETS '09, 115--122.
23. Shaun K. Kane, Jacob O. Wobbrock, and Richard E. Ladner. 2011. Usable gestures for blind people: understanding preference and performance. Proc. CHI '11, 413--422.
24. Frederic Kerber, Tobias Kiefer, and Markus Löchtefeld. 2016. Investigating Interaction Techniques for State-of-the-Art Smartwatches. Proc. CHI EA '16, 2540--2547.
25. Christian Kray, Daniel Nesbitt, John Dawson, and Michael Rohs. 2010. User-defined gestures for connecting mobile phones, public displays, and tabletops. Proc. MobileHCI '10, 239--248.
26. Meethu Malu and Leah Findlater. 2015. Personalized, Wearable Control of a Head-mounted Display for Users with Upper Body Motor Impairments. Proc. CHI '15, 221--230.
27. V Mathiowetz, G Volland, N Kashman, and K Weber. 1985. Adult norms for the Box and Block Test of manual dexterity. The American journal of occupational therapy, 6: 386--91.
28. Matthias Mielke and Rainer Brück. 2015. A Pilot Study about the Smartwatch as Assistive Device for Deaf People. Proc. ASSETS '15, 301--302.
29. Kyle Montague, Hugo Nicolau, and Vicki L. Hanson. 2014. Motor-impaired touchscreen interactions in the wild. Proc. ASSETS '14, 123--130.
30. Maia Naftali and Leah Findlater. 2014. Accessibility in context: understanding the truly mobile experience of smartphone users with motor impairments. Proc. ASSETS '14, 209--216.
31. Jerome Pasquero, Scott J. Stobbe, and Noel Stonehouse. 2011. A haptic wristwatch for eyes-free interactions. Proc. CHI '11, 3257--3266.
32. Lorenzo Porzi, Stefano Messelodi, Carla Mara Modena, and Elisa Ricci. 2013. A smart watch-based gesture recognition system for assisting people with visual impairments. Proc. IMMPD '13, 19--24.
33. R. Raya, J.O. Roa, E. Rocon, R. Ceres, and J.L. Pons. 2010. Wearable inertial mouse for children with physical and cognitive impairments. Sensors and Actuators A: Physical 162, 2: 248--259.
34. Steven Schirra and Frank R. Bentley. 2015. "It's kind of like an extra screen for my phone": Understanding Everyday Uses of Consumer Smart Watches. Proc. CHI EA '15, 2151--2156.
35. Shari Trewin, Cal Swart, and Donna Pettick. 2013. Physical accessibility of touchscreen smartphones. Proc. ASSETS '13, 1--8.
36. Chat Wacharamanotham, Jan Hurtmanns, Alexander Mertens, Martin Kronenbuerger, Christopher Schlick, and Jan Borchers. 2011. Evaluating swabbing: a touchscreen input method for elderly users with tremor. Proc. CHI '11, 623--626.
37. Jacob O. Wobbrock, Meredith Ringel Morris, and Andrew D. Wilson. 2009. User-defined gestures for surface computing. Proc. CHI '09, 1083--1092.
38. Jacob O. Wobbrock, Brad A. Myers, and John A. Kembel. 2003. EdgeWrite: a stylus-based text entry method designed for high accuracy and stability of motion. Proc. UIST '03, 61--70.
39. Robert Xiao, Gierad Laput, and Chris Harrison. 2014. Expanding the input expressivity of smartwatches with mechanical pan, twist, tilt and click. Proc. CHI '14, 193--196.
40. Huiyu Zhou and Huosheng Hu. 2008. Human motion tracking for rehabilitation-A survey. Biomedical Signal Processing and Control 3, 1: 1--18.

Index Terms

• Human-centered computing

• Social and professional topics