Evaluation of haptically augmented touchscreen gui elements under cognitive load

ABSTRACT

Adding expressive haptic feedback to mobile devices has great potential to improve their usability, particularly in multitasking situations where one's visual attention is required. Piezoelectric actuators are emerging as one suitable technology for rendering expressive haptic feedback on mobile devices. We describe the design of redundant piezoelectric haptic augmentations of touchscreen GUI buttons, progress bars, and scroll bars, and their evaluation under varying cognitive load. Our haptically augmented progress bars and scroll bars led to significantly faster task completion, and favourable subjective reactions. We further discuss resulting insights into designing useful haptic feedback for touchscreens and highlight challenges, including means of enhancing usability, types of interactions where value is maximized, difficulty in disambiguating background from foreground signals, tradeoffs in haptic strength vs. resolution, and subtleties in evaluating these types of interactions.

References

1. Brewster, S. A. and Brown, L. M. Tactons: Structured tactile messages for non-visual information display. In Proc. of Australasian User Interface Conf. Australian Computer Society, 2004, 15--23.
2. Chan, A., MacLean, K. and McGrenere, J. Learning and identifying haptic icons under workload. In Proc. of the 1st Joint Eurohaptics Conf. and Symp. on Haptic Interfaces for Virtual Environment and Teleoperator Sys. (WHC 2005). Pisa, Italy, 2005, 432--439.
3. Chang, A., O'Modhrain, S., Jacob, R., Gunther, E. and Ishii, H. ComTouch: Design of a vibrotactile communication device. In Proc. of the Conf. on Designing interactive systems (DIS '02). ACM Press, NY, NY, 2002, 312--320.
4. Dennerlein, J. T., Martin, D. B. and Hasser, C. Force-feedback improves performance for steering and combined steering-targeting tasks. In Proc. of CHI '00. ACM Press, NY, NY, 2000, 423--429.
5. Fukumoto, M. and Sugimura, T. Active Click: Tactile feedback for touch panels. In Extended Abstracts of CHI '01. ACM Press, NY, NY, 2001, 121--122.
6. Heeger, D. Signal Detection Theory. http://www.cns.nyu.edu/~david/sdt/sdt.html
7. Kaaresoja, T., Brown, L. M. and Linjama, J. Snap-Crackle-Pop: Tactile feedback for mobile touch screens. In Proc of EuroHaptics 2006, 2006
8. Laitinen, P. and Mäenpää, J. Enabling mobile haptic design: Piezoelectric actuator technology properties in hand held devices. In IEEE Intl Workshop on Haptic Audio Visual Environments and their Applications. 2006, 40--43.
9. Luk, J., Pasquero, J., Little, S., MacLean, K., Levesque, V. and Hayward, V. A role for haptics in mobile interaction: Initial design using a handheld tactile display prototype. In Proc. of CHI '06. ACM Press, NY, NY, 2006, 171--180.
10. Oakley, I., McGee, M. R., Brewster, S. and Gray, P. Putting the feel in 'look and feel'. In Proc. of CHI'2000. ACM Press, NY, NY, 2000, 415--422.
11. Oulasvirta, A., Tamminen, S., Roto, V. and Kuorelahti, J. Interaction in 4-second bursts: The fragmented nature of attentional resources in mobile HCI. In Proc of CHI '05. ACM Press, NY, NY, 2005, 919--928.
12. Oviatt, S., Coulston, R. and Lunsford, R. When do we interact multimodally?: Cognitive load and multimodal communication patterns. In Proc. of ICMI '04: ACM Press, NY, NY, 2004, 129--136.
13. Poupyrev, I. and Maruyama, S. Tactile interfaces for small touch screens. In Proc. of UIST '03. ACM Press, NY, NY, 2003, 217--220.
14. Poupyrev, I., Maruyama, S. and Rekimoto, J. Ambient touch: Designing tactile interfaces for handheld devices. In Proc of UIST '02. ACM Press, NY, NY, 2002, 51--60.
15. Poupyrev, I., Okabe, M. and Maruyama, S. Haptic feedback for pen computing: Directions and strategies. In Extended Abstracts of CHI '04. ACM Press, NY, NY, 2004, 1309--1312.
16. Shedden, J Signal Detection Theory. http://brain.mcmaster.ca/~lab/SDT/dprime.html
17. Tikka, V. and Laitinen, P. Designing haptic feedback for touch display: Experimental study of perceived intensity and integration of haptic and audio. In 1st Intl Workshop on Haptic and Audio Interaction Design. Springer, Berlin/Heidelberg, 2006, 36--44.

Index Terms

• Human-centered computing

  • Human computer interaction (HCI)

    • Interaction devices

      • Haptic devices

    • Interaction paradigms

      • Graphical user interfaces