Mapping information to audio and tactile icons

ABSTRACT

We report the results of a study focusing on the meanings that can be conveyed by audio and tactile icons. Our research considers the following question: how can audio and tactile icons be designed to optimise congruence between crossmodal feedback and the type of information this feedback is intended to convey? For example, if we have a set of system warnings, confirmations, progress up-dates and errors: what audio and tactile representations best match the information or type of message? Is one modality more appropriate at presenting certain types of information than the other modality? The results of this study indicate that certain parameters of the audio and tactile modalities such as rhythm, texture and tempo play an important role in the creation of congruent sets of feedback when given a specific type of information to transmit. We argue that a combination of audio or tactile parameters derived from our results allows the same type of information to be derived through touch and sound with an intuitive match to the content of the message.

References

1. Bernsen, N. O., "Modality Theory in Support of Multimodal Interface Design," ERCIM Workshop on Multimodal Human-Computer Interaction, pp. 27 -- 44, 1993.
2. Blattner, M. M., Sumikawa, D. A., and Greenberg, R. M., "Earcons and Icons: Their Structure and Common Design Principles," Human Computer Interaction, vol. 4(1), pp. 11 -- 44, 1989.
3. Brewster, S. A., Wright, P. C., and Edwards, A. D. N. Ex-perimentally Derived Guidelines for the Creation of Earcons. In Proc HCI '95, Springer (1995), pp. 155 -- 159.
4. Brown, L. M. and Brewster, S. A. Multidimensional Tactons for Non-Visual Information Display in Mobile Devices. In Proc MobileHCI '06, ACM Press (2006), pp. 231 -- 238.
5. Chang, A. and O'Sullivan, C. Audio-haptic feedback in mobile phones. In Proc CHI '05 extended abstracts on Human factors in computing systems, ACM Press (2005).
6. Edworthy, J., Hellier, E. J., and Hards, R., "The Semantic Associations of Acoustic Parameters Commonly Used in the Design of Auditory Information and Warning Signals," Ergonomics, vol. 38(11), pp. 2341 -- 2361, 1995.
7. Enriquez, M., MacLean, K., and Chita, C. Haptic Phenomes: Basic Building Blocks of Haptic Communication. In Proc 8th International Conference on Multimodal Interfaces (ICMI'06), IEEE (2006), pp. 302 -- 309.
8. Gaver, W., "Auditory Icons: Using Sound in Computer Interfaces," ACM SIGCHI Bulletin, vol. 19(1), p. 74, 1987.
9. Geldard, F. A., "Some Neglected Possibilities of Communication," Science, vol. 131(3413), pp. 1583 -- 1588, 1960.
10. Hoggan, E. and Brewster, S. A. Designing audio and tactile crossmodal icons for mobile devices. In Proc Proceedings of the 9th international conference on Multimodal interfaces, ACM Press (2007), pp. 162 -- 169.
11. Hoggan, E. and Brewster, S. A. New Parameters for Tacton Design. In Proc Vol II CHI 2007, ACM Press (2007), pp. 2417 -- 2422.
12. Koskinen, E., Kaaresoja, T., and Laitinen, P. Feel-Good Touch: Finding the Most Pleasant Tactile Feedback for a Mobile Touch Screen Button. In Proc ICMI 2008, ACM Press (2008), pp. 297 -- 304.
13. Lewkowicz, D. J., "Development of Intersensory Perception in Human Infants," in The Development of Intersensory Perception: Comparative Perspectives. Eds. D. J. Lewkowicz and R. Lickliter. Hillsdale, N.J.: Erlbaum, 1994, pp. 165 -- 203.
14. MacLean, K. and Enriquez, M. Perceptual Design of Haptic Icons. In Proc Eurohaptics, 2003, pp. 351 -- 363.
15. McGookin, D. and Brewster, S. A., "Understanding Concurrent Earcons: Applying Auditory Scene Analysis Principles to Concurrent Earcon Recognition," ACM Transactions on Applied Perception, vol. 1, pp. 130 -- 155, 2004.
16. Mortimer, B., Zets, G., and Cholewiak, R. W., "Vibrotactile Transduction and Transducers," The Journal of the Acoustic Society of America, vol. 121, pp. 2970 -- 2977, 2007.
17. Palomäki, H. Meanings Conveyed by Simple Auditory Rhythms. In Proc ICAD'06, 2006.
18. Patterson, R. D., Guidelines for Auditory Warning Systems on Civil Aircraft, C.A.A. Paper 82017, C. A. Authority (1982).
19. Poupyrev, I. and Maruyama, S. Tactile Interfaces for Small Touch Screens. In Proc 16th Annual ACM Symposium on User Interface Software and Technology, ACM Press (2003), pp. 217 -- 220.
20. Rovers, A. F. and van Essen, H. A. Using Haptic Feedback in Everyday Products. In Proc EuroHaptics, Springer-Verlag (2006), pp. 447 -- 453.
21. Shneiderman, B., Designing the User Interface. Reading, MA: Addison--Wesley, 1998.
22. van Erp, J. B. Tactile Navigation Display. In Proc First international Workshop on Haptic Human -- Computer Interaction, LNCS (2001), pp. 165 -- 173.
23. van Erp, J. B. and van Veen, H. A. H. C. Vibrotactile information presentation in automobiles. In Proc EuroHaptics 2001, Springer Verlag (2001), pp. 99 -- 104.

Index Terms

• Human-centered computing

  • Human computer interaction (HCI)

    • Interaction devices

      • Haptic devices

    • Interaction techniques

      • Auditory feedback