A perceptual framework for the auditory display of scientific data

ABSTRACT

A principal requirement of the auditory display of scientific data is an accurate portrayal of the information contained in the data. The characteristics of the display device can significantly affect the faithfulness of the data presentation. Human hearing is a complex nonlinear process and the intuitive comprehension of the display requires consideration of perceptual interactions and the natural connection between the data and the perception of the data. This chapter proposes a perceptual framework for observable and systematic specification and comprehension of sounds in a device-independent auditory display of scientific data. The framework consists of a perceptually scaled sound space and a display model that maps points from the perceptual space to a display device parameter space while preserving the interrelationships between them. The advantages of a perceptual space are described using the example of an established framework for applying perceptually uniform color models to scientific visualization. A perceptual sound space is defined and constructed by scaling a naturally ordered sound model derived from research studies in timbre perception. An auditory display is implemented on a Sun Sparc10 workstation using Csound and samples of musical instruments. The display model consists of an interpolated mapping from perceptual space to display space and a description of the display gamut that is the boundary in perceptual space between points that the display can realize and those that it cannot. The display gamut can be used to analyze the display, compare displays, and optimize data mapping sequences for the display. The concepts are illustrated by a graphic visualization of the gamut of the implemented display as a geometric shape in the perceptual sound space.

REFERENCES

• 1: Barrass, S. 1994. A naturally ordered geometric model of sound inspired by colour theory. In Proceedings of Synaesthetica '94. Australian Centre for Arts and Technology, Canberra.
• 2: Bertin, J. 1981. Graphics and Graphic Information Processing. Walter de Gruyter, Berlin.
• 3: Bone, D. J. 1993. Adaptive color-printer modelling using regularized linear splines. In Proceedings of Device Independent Color Imaging, IS&T/SPIE Symposium on Electronic Imaging Science and Technology.
• 4: Bregman, A. S. 1990. Auditory Scene Analysis: The Perceptual Organization of Sound. MIT Press, Cambridge, MA.
• 5: Caivano, J. L. 1994. Color and sound: Physical and psychophysical relations. COLOR Research and Applications 19, 2, 126--132.
• 6: Gaver, W. 1986. Auditory icons. Human-Computer Interaction, 2(2) Lawrence Erlbaum Assoc., Mahwah, NJ.
• 7: Gibson, J. J. 1966. The Senses Considered as Perceptual Systems. Houghton Mifflin, Boston, MA.
• 8: Grey, J. M. 1975. Exploration of musical timbre. Ph.D. Thesis, CCRMA Dept. of Music, Stanford University, Report No. STAN-M-2.
• 9: Handel, S. 1989. Listening: An Introduction to the Perception of Acoustic Events. MIT Press, Cambridge, MA.
• 10: Kaplan, B. and Maccuish, J. 1993. User Documentation---Sonify module for AVS, Center for Innovative Computer Applications (CICA), Indiana University.
• 11: Kramer, G. 1994. Some organizing principles for representing data with sound. Auditory Display: Sonification, Audification, and Auditory Interfaces. G. Kramer, Ed. Santa Fe Institute Studies in the Sciences of Complexity, Proc. Vol. XVIII. Addison-Wesley, Reading, MA.
• 12: Padgham, C. 1986. The scaling of the timbre of the pipe organ. Acustica 60, 3, 189--204.
• 13: Pridmore, R. W. 1992. Music and color: Relations in the psychophysical perspective. COLOR Research and Applications 17, 1, 57--61.
• 14: Philip K. Robertson, Visualizing Color Gamuts: A User Interface for the Effective Use of Perceptual Color Spaces in Data Displays, IEEE Computer Graphics and Applications, v.8 n.5, p.50-64, September 1988 [doi>10.1109/38.7761 ]
• 15: Scaletti, C. 1994. Sound synthesis algorithms for auditory data representations. In Auditory Display: Sonification, Audification, and Auditory Interfaces. G. Kramer, Ed. Santa Fe Institute Studies in the Sciences of Complexity, Proc. Vol. XVIII. Addison-Wesley, Reading, MA.
• 16: Sebba, R. 1991. Structural correspondence between music and color. COLOR Research and Application 16, 2, 81--88.
• 17: Slawson, A. W. 1968. Vowel quality and musical timbre as functions of spectrum envelope and fundamental frequency. Journal of the Acoustical Society of America 43, 87--101.
• 18: Smith, S. 1990. Representing data with sound. Proceedings of the Visualization 1990 Conference. IEEE Computer Society Press.
• 19: Vercoe, B. 1991. CSOUND, A Manual for the Audio Processing System and Supporting Programs. MIT Media Laboratory, Cambridge, MA.
• 20: von Bismarck, G. 1974. Timbre of steady state sounds: A factorial investigation of its verbal attributes. Acustica 30, 146--159.
• 21: Ware, C. 1993. The foundations of experimental semiotics: A theory of sensory and conventional representation. Journal of Visual Language. & Computing 4, 91--100.
• 22: Wyszecki, G. and Stiles. W. S. 1967. Color Science. Wiley Sons, New York.
• 23: Zwicker, E. and Fastl. H. 1990. Psychoacoustics Facts and Models. Springer-Verlag, Berlin.

INDEX TERMS

General Terms:
Experimentation, Human Factors, Performance

Keywords:
Sonification, auditory display, data mapping