Exploring and Understanding Unintended Touch during Direct Pen Interaction

Abstract

The user experience on tablets that support both touch and styli is less than ideal, due in large part to the problem of unintended touch or palm rejection. Devices are often unable to distinguish between intended touch (i.e., interaction on the screen intended for action) and unintended touch (i.e., incidental interaction from the palm, forearm, or fingers). This often results in stray ink strokes and accidental navigation, frustrating users. We present a data collection experiment where participants performed inking tasks, and where natural tablet and stylus behaviors were observed and analyzed from both digitizer and behavioral perspectives. An analysis and comparison of novel and existing unintended touch algorithms revealed that the use of stylus information can greatly reduce unintended touch. Our analysis also revealed many natural stylus behaviors that influence unintended touch, underscoring the importance of application and ecosystem demands, and providing many avenues for future research and technological advancement.

References

1. Tulio de Souza Alcantara, Jennifer Ferreira, and Frank Maurer. 2013. Interactive Prototyping of Tabletop and Surface Applications. In Proceedings of the ACM SIGCHI Symposium on Engineering Interactive Computing Systems. 229--238.
2. Michelle Annett. 2014a. The Fundamental Issues of Pen-Based Interaction with Tablet Devices. Doctoral Dissertation, University of Alberta, Edmonton, Alberta, Canada.
3. Michelle Annett, Fraser Anderson, Anoop Gupta, and Walter F. Bischof. 2014b. The Pen Is Mightier: Understanding Stylus Behavior While Inking on Tablets. In Proceedings of Graphics Interface. 193--200.
4. Michelle Annett, Tovi Grossman, Daniel Wigdor, and George Fitzmaurice, 2011. Medusa: A Proximity-Aware Multi-Touch Tabletop. In Proceedings of the ACM Symposium on User Interfaces and Software Technology. 337--346.
5. Michelle Annett, Albert Ng, Paul Dietz, Anoop Gupta, and Walter F. Bischof. 2014c. How Low Should We Go? Understanding the Perception of Latency While Inking. In Proceedings of Graphics Interface. 167--174.
6. Atmel. 2014. Revolutionizing the Touch User Experience. Retrieved June 2014 from http://www.atmel.com/microsite/stylus/default.aspx.
7. Peter Brandl, Jakob Letiner, Thomas Seifried, Michael Haller, Bernard Doray, and Paul To. 2009. Occlusion-Aware Menu Design for Digital Tabletops. In Extended Abstracts of the SIGCHI Conference on Human Factors in Computing Systems. 3223--3228.
8. William S. Buxton. 1990. A Three-State Model of Graphical Input. In Proceedings of the IFIP TC13 3rd Interaction Conference on Human-Computer Interaction. 449--456.
9. Xiang Cao, Andrew Wilson, Ravin Balakrishnan, Ken Hinckley, and Scott Hudson. 2008. ShapeTouch: Leveraging Contact Shape on Interactive Surfaces. In Proceedings of the 3rd IEEE International Workshop on Horizontal Interactive Human Computer Systems. 129--136.
10. Devin Coldewey. 2011. Synaptics New Touchscreens Can Detect the Head of a Pin. Retrieved from http://techcrunch.com/2011/02/15/synaptics-new-touchscreens-can-detect-the-head-of-a-pin/.
11. Chi Tai Dang, Martin Straub, and Elisabeth André. 2009. Hand Distinction for Multi-Touch Tabletop Interaction. In Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces. 101--108.
12. Philipp Ewerling, Alenander Kulik, and Bernd Froehlich. 2012. Finger and Hand Detection for Multi-Touch Interfaces Based on Maximally Stable Extremal Regions. In Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces. 173--182.
13. George Fitzmaurice, Ravin Balakrishnan, Gordon Kurtenbach, and William S. Buxton. 1999. An Exploration into Supporting Artwork Orientation in the User Interface. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 167--174.
14. Jens Gerken, Hans-Christian Jetter, Toni Schmidt, and Harald Reiterer. 2010. Can “Touch” Get Annoying? In Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces. 257--258.
15. Yves Guiard. 1987. Asymmetric Division of Labor in Human Skilled Bi-manual Action: The Kinematic Chain as a Model. Journal of Motor Behavior. 19, 4, 486--517.
16. Mark Hancock and Kellogg Booth. 2004. Improving Menu Placement Strategies for Pen Input. In Proceedings of Graphics Interface. 221--230.
17. Ken Hinckley and Daniel Wigdor. 2012. Input Technologies and Techniques. The Human Computer Interaction Handbook: Fundamentals, Evolving Technologies, and Emerging Applications (3rd ed). CRC Press, Boca Raton, FL.
18. Ken Hinckley, Xiang Chen, and Hrvoje Benko. 2013. Motion and Context Sensing Techniques for Pen Computing. In Proceedings of Graphics Interface. 71--78.
19. Ken Hinckley, Koji Yatani, Michel Pahud, Nicole Coddington, Jenny Rodenhouse, Andy Wilson, Hrvoje Benko, and William S. Buxton. 2010a. Pen + Touch = New Tools. In Proceedings of the ACM Symposium on User Interfaces and Software Technology. 27--36.
20. Ken Hinckley, Koji Yatani, Michel Pahud, Nicole Coddington, Jenny Rodenhouse, Andy Wilson, Hrvoje Benko, and William S. Buxton 2010b. Manual Deskterity: An Exploration of Simultaneous Pen + Touch Direct Input. In Extended Abstracts on Human Factors in Computing Systems. 2973--2802.
21. Ricardo Jota, Albert Ng, Paul Dietz, and Daniel Wigdor. 2013. How Fast Is Fast Enough? A Study of the Effects of Latency in Direct-touch Pointing Tasks. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 2291--2300.
22. Beverly Laundry. 2011. Sheet Music Unbound: A Fluid Approach to Sheet Music Display and Annotation on A Multi-Touch Screen. Master's thesis, University of Waikato, Hamilton, New Zealand.
23. Jerry Levy and Marylou Reid. 1978. Variations in Cerebral Organization as a Function of Handedness, Hand Posture in Writing, and Sex. Journal of Experimental Psychology 107, 2 119--144.
24. Chin-Lung Lin, Chia-Sheng Li, Yi-Ming Chang, Tsung-Chih Lin, Jiann-Fuh Chen, and U.-Chen Lin. 2013. Pressure Sensitive Stylus and Algorithm for Touchscreen Panel. Journal of Display Technology 9, 1, 17--23.
25. Nicolai Marquardt, Johannes Kiemer, David Ledo, Sebastian Boring, and Saul Greenberg. 2011. Designing User-, Hand-, and Handpart-Aware Tabletop Interactions with the TouchID Toolkit. In Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces. 21--30.
26. Fabrice Matulic and Moria Norrie. 2012. Empirical Evaluation of Uni- and Bimodal Pen and Touch Interaction Properties on Digital Tabletops. In Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces. 143--152.
27. Microsoft. 2013. Microsoft Grows Surface Family. Retrieved from http://www.microsoft.com/en-us/news/press/2013/jan13/01-22surfacefamilypr.aspx.
28. Microsoft. 2014. Windows Hardware Certification Requirements. Retrieved from http://msdn.microsoft.com/en-us/library/windows/hardware/dn423132.
29. Sundar Murugappana, Vinayak, Niklas Elmqvist, and Karthik Ramani. 2012. Extended Multitouch: Recovering Touch Posture and Differentiating Users Using a Depth Camera. In Proceedings of the ACM Symposium on User Interfaces and Software and Technology. 487--496.
30. Albert Ng, Michelle Annett, Paul Dietz, Anoop Gupta, and Walter F. Bischof. 2014. In the Blink of An Eye: Investigating Latency Perception During Stylus Interaction. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 1103--1112.
31. Albert Ng, Julian Lepinski, Daniel Wigdor, Steven Sanders, and Paul Dietz. 2012. Designing for Low-Latency Direct-Touch Input. In Proceedings of the ACM Symposium on User Interfaces and Software Technology. 453--462.
32. R. C. Oldfield. 1971. The Assessment and Analysis of Handedness: The Edinburgh Inventory. Journal of Neuropsychologia 9, 97--113.
33. Openframeworks. 2009. Ofxtouch Is a Multitouch Add-On for Openframeworks. Retrieved June 2013 from https://code.google.com/p/ofxtouch/.
34. Raf Ramakers, Davy Vanacken, Kris Luyten, Karin Coninx, and Johannes Schöning. 2012. Carpus: A Non-Intrusive User Identification Technique for Interactive Surfaces. In Proceedings of the ACM Symposium on User Interfaces and Software Technology. 35--44.
35. Simon Sage. 2011. Synaptics Series 4 Capacitive Touchscreen Works With Stylus, Through Gloves. Retrieved from http://www.intomobile.com/2011/02/16/synaptics-series-4-clearpad/.
36. Samsung. 2013. How Do I Use Air Gestures? Retrieved July 2013 from http://www.samsung.com/us/support/howtoguide/N0000003/10141/120552.
37. Smartkeitai. 2013. Sharp. 10.1-Inch IGZO High Sensitivity Tablet Display W. Pen Input. Retrieved from http://www.youtube.com/watch?v=ofvzmluaibg.
38. Julie Schwartz, Robert Xiao, Jennifer Mankoff, Scott Hudson, and Chris Harrison. 2014. Probabilistic Palm Rejection Using Spatiotemporal Touch Features and Iterative Classification. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 2009--2012.
39. SmudgeGuard. (n.d.). Eliminates Smudges and Hand Friction for Artists, Left-Handed Children & Adults, Wacom and Tablet PC Users. Retrieved June 2014 from http://smudgeguard.com.
40. Ke Shu. 2013. Understanding and Rejecting Errant Touches on Multi-touch Tablets. Master's thesis, Singapore Management University, Bras Basah, Singapore.
41. Itiro Siio and Hitmoi Tsujita. 2006. Mobile Interaction Using Paperweight Metaphor. In Proceedings of the ACM Symposium on User Interfaces and Software Technologies. 111--114.
42. Hyunyoung Song, Hrvoje Benko, François Gumbretière, Shahram Izadi, Xiang Cao, and Ken Hinckley. 2011. Grips and Gestures on a Multi-touch Pen. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 1323--1332.
43. Sony 2014. Floating Touch-Developer World Mobile. Retrieved June 2014 from http://developer.sonymobile.com/knowledge-base/technologies/floating-touch/.
44. STMicroelectronics. 2014. Touchscreen Controllers. Retrieved June 2014 from http://www.st.com/web/en/catalog/sense_power/FM89/SC1717?s_searchtype = keyword.
45. Wacom. 2014. Inkling. Retrieved June 2014 from http://www.wacom.com/en/ca/creative/inkling.
46. Julie Wagner, Stéphane Huot, and Wendy Mackay. 2012. Bitouch and Bipad: Designing Bi-manual Interaction for Hand-Held Tablets. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 2317--2326.
47. Feng Wang, Xiang Cao, Xiangshi Ren, and Pourang Irani. 2009. Detecting and Leveraging Finger Orientation for Interaction with Direct-Touch Surfaces. In Proceedings of the ACM Symposium on User Interfaces and Software Technology. 23--32.
48. Daniel Wigdor, Hrvoje Benko, John Pella, Jarrod Lombardo, and Sarah Williams. 2011. Rock & Rails: extending multi-touch interactions with shape gestures to enable precise spatial manipulations. In Proceedings of SIGCHI Conference on Human Factors in Computing Systems. 1581--1590.
49. Michael Wu and Ravin Balakrishnan. 2003. Multi-Finger and Whole Hand Gestural Interaction Techniques for Multi-User Tabletop Displays. In Proceedings of the ACM Symposium on User Interfaces and Software Technology. 193--202.
50. Elba del Carmen Valderrama Bahamónde, Thomas Kubitza, Niels Henze, and Albrecht Schmidt. 2013. Analysis of Children's Handwriting on Touchscreen Phones. In Proceedings of the International Conference on Human-Computer Interaction with Mobile Devices and Services. 171--174.
51. Daniel Vogel and Ravin Balakrishnan. 2010a. Occlusion-Aware Interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 263--272.
52. Daniel Vogel and Ravin Balakrishnan. 2010b. Direct Pen Interaction with a Conventional Graphic User Interface. Human Computer Interaction 25, 4, 324--388.
53. Daniel Vogel, Matthew Cudmore, Géry Casiez, Ravin Balakrishnan, and Liam Keliher. 2009. Hand Occlusion with Tablet-Sized Direct Pen Input. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 557--566
54. Xiaoqi Yan, Peng Song, Chi-Wing Fu, Wooi Boon Goh, and Kwan-Liu Ma. 2013. Exploring Volume Visualizations with Whole-hand Multi-touch Gestures. In Proceedings of Pacific Graphics.
55. Dongwook Yoon, Nicholas Chen, and François Gumbretière. 2013. TextTearing for Easy Annotaiton of Digital Documents. In Proceedings of the ACM Symposium on User Interfaces and Software Technology. 107--112.
56. Robert Zeleznik, Timothy Miller, Andries van Dam, Chuanjun Li, Dana Tenneson, Christopher Maloney, Joseph LaViola Jr. 2008. Applications and Issues in Pen-Centric Computing. MultiMedia 15, 4, 14--21.
57. Hong Zhang, Xing-Dong Yang, Barrett End, Hai-Ning Liang, Pierre Boulanger, and Pourang Irani. 2012. See Me, See You: A Lightweight Method for Discriminating User Touches on Tabletop Displays. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 2327--2336.

Index Terms

• Human-centered computing

  • Human computer interaction (HCI)