Programming and enjoying music with your eyes closed

ABSTRACT

Design and user evaluation of a multimodal interaction style for music programming is described. User requirements were instant usability and optional use of a visual display. The interaction style consists of a visual roller metaphor. User control of the rollers proceeds by manipulating a force feedback trackball. Tactual and auditory cues strengthen the roller impression and support use without a visual display. The evaluation investigated task performance and procedural learning when performing music programming tasks with and without a visual display. No procedural instructions were provided. Tasks could be completed successfully with and without a visual display, though programming without a display needed more time to complete. Prior experience with a visual display did not improve performance without a visual display. When working without a display, procedures have to be acquired and remembered explicitly, as more procedures were remembered after working without a visual display. It is demonstrated that multimodality provides new ways to interact with music.

REFERENCES

• 1: Consumer Reports (1997). Jukebox time? Consumer Reports, 62, 2, February 1997, 34-38.
• 2: Kumin, D., (1994). Ch-ch-changers! Stereo Review, 59, 60- 66.
• 3: J. H. Eggen , R. Haakma , J. H. D. M. Westerink, Layered protocols: hands-on experience, International Journal of Human-Computer Studies, v.44 n.1, p.45-72, Jan. 1996 [doi>10.1006/ijhc.1996.0003 ]
• 4: Veer, G.C., van der (1990). Human computer interaction: learning, individual differences, and design recommendations. Doctoral Thesis, Vrije Universiteit, Amsterdam, the Netherlands.
• 5: Hutchins, E. (1989). Metaphors for Interface Design. In: Taylor, M.M., N6el, F., and Bouwhuis, D.G. (Eds.). The Structure of Multimodal Dialogue, Amsterdam: North- Holland, 11-28.
• 6: Kieras, D.E., and Bovair, S. (1984). The role of a mental model in learning to operate a device. Cognitive Science, 8, 255-273.
• 7: Payne, S.J., (1988). Metaphorical instruction and the early learning of an abbreviated-command computer system, Acta Psychologica, 69, 207-230.
• 8: Jeff Shrager , David Klahr, Instructionless learning about a complex device: the paradigm and observations, International Journal of Man-Machine Studies, v.25 n.2, p.153-189, Aug. 1986 [doi>10.1016/S0020-7373(86)80075-X]
• 9: Frits L. Engel , Peter Goossens , Reinder Haakma, Improved efficiency through I- and E-feedback: a trackball with contextual force feedback, International Journal of Human-Computer Studies, v.41 n.6, p.949-974, Dec. 1994 [doi>10.1006/ijhc.1994.1087 ]
• 10: I. Scott MacKenzie , Abigail Sellen , William A. S. Buxton, A comparison of input devices in element pointing and dragging tasks, Proceedings of the SIGCHI conference on Human factors in computing systems: Reaching through technology, p.161-166, April 27-May 02, 1991, New Orleans, Louisiana, United States [doi>10.1145/108844.108868]
• 11: Gaver, W. (1993). What in the world do we hear? An ecological approach to auditory event perception. Ecological Psychology, 5, 4, 285-313.
• 12: Hermes, D.J. (1998). Auditory material perception. IPO Annual Progress Report, 33, 95-102.
• 13: James Rumbaugh , Ivar Jacobson , Grady Booch, The Unified Modeling Language reference manual, Addison-Wesley Longman Ltd., Essex, UK, 1998
• 14: Pauws, S.C. (2000). Music and choice: Adaptive systems and multimodal interaction, Doctoral Thesis, Eindhoven University of Technology, the Netherlands.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.1 MODELS AND PRINCIPLES

Additional Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)

General Terms:
Design, Human Factors, Management, Performance, Theory

Keywords:
interactive music system, interface design, multimodal interaction, nonvisual interaction, user evaluation