Designing spatial audio interfaces to support multiple audio streams

ABSTRACT

Auditory interfaces offer a solution to the problem of effective eyes-free mobile interactions. However, a problem with audio, as opposed to visual displays, is dealing with multiple simultaneous outputs. Any audio interface needs to consider: 1) simultaneous versus sequential presentation of multiple audio streams, 2) 3D audio techniques to place sounds in different spatial locations versus a single point of presentation, 3) dynamic movement versus fixed locations of audio sources. We present an experiment using a divided-attention task where a continuous podcast and an audio menu compete for attention. A sequential presentation baseline assessed the impact of cognitive load, and as expected, dividing attention had a significant effect on overall performance. However, spatial audio still increased the users' ability to attend to two streams, while dynamic movement of streams led to higher perceived workload. These results will provide guidelines for designers when building eyes-free auditory interfaces for mobile applications.

References

1. Li, K. A., Baudisch, P. and Hinckley, K. 2008. Blindsight: eyes-free access to mobile phones. In Proc. CHI 2008 (Florence, Italy, 2008). ACM Press, 1389--1398.
2. Cherry, E. C. 1953. Some experiments on the recognition of speech, with one and with two ears. Journal of Acoustical Society of America 25(5), (1953) 975--979.
3. Begault, D.R. 1994. 3D Sound for Virtual Reality and Multimedia. Academic Press Cambridge.
4. Schmandt, C., Mullins, A. 1995. AudioStreamer: exploiting simultaneity for listening. In Proc. CHI 1995, ACM Press (1995), 218--219.
5. Walker, A., Brewster, S.A., McGookin, D. and Ng, A. 2001. Diary in the sky: A spatial audio display for a mobile calendar. In Proc. BCS IHM-HCI 2001, Springer (2001), 531--540.
6. Stifelman, L.J. 1994. The cocktail party effect in auditory interfaces: a study of simultaneous presentation. MIT Media Laboratory Technical Report.
7. Ihlefeld, A. and Shinn-Cunningham, B. G. 2008. Spatial release from energetic and informational masking in a divided speech identification task. Journal of the Acoustic Society of America, 123 (2008) 4380--4392.
8. Vazquez-Alvarez, Y. and Brewster, S.A. 2009. Investigating Background & Foreground Interactions Using Spatial Audio Cues. In Proc. CHI 2009, ACM Press (2009), 3823--3828.
9. www.nseries.com/index.html#l=products,n95_8gb
10. http://theoreticlabs.com/dev/api/jsr-234/javax/microedition/amms/package-summary.html
11. Hart, S. G. and Staveland, L. E. 1988. Development of Nasa Tlx (Task Load Index): Results of Empirical and Theoretical Research. Human Mental Workload (1988), 139--183.
12. Dietz, P.H. and Yerazunis, W.S. 2001. Real-Time Audio Buffering for Telephone Applications. In Proceedings of UIST: ACM Symposium on User Interface Software and Technology (Orlando FL USA, November 2001), ACM Press, 193--194.