Gil Weinberg
ABSTRACT
We present new developments in the improvisational robotic percussionist project, aimed at improving human-robot interaction through design, mechanics, and perceptual modeling. Our robot, named Haile, listens to live human players, analyzes perceptual aspects in their playing in real-time, and uses the product of this analysis to play along in a collaborative and improvisatory manner. It is designed to combine the benefits of computational power in algorithmic music with the expression and visual interactivity of acoustic playing. Haile's new features include an anthropomorphic form, a linear-motor based robotic arm, a novel perceptual modeling implementation, and a number of new interaction schemes. The paper begins with an overview of related work and a presentation of goals and challenges based on Haile's original design. We then describe new developments in physical design, mechanics, perceptual implementation, and interaction design, aimed at improving human-robot interactions with Haile. The paper concludes with a description of a user study, conducted in an effort to evaluate the new functionalities and their effectiveness in facilitating expressive musical human-robot interaction. The results of the study show correlation between human's and Haile's rhythmic perception as well as user satisfaction regarding Haile's perceptual and mechanical abilties. The study also indicates areas for improvement such as the need for better timbre and loudness control and more advance and responsive interaction schemes.
- Baginsky, N.A. The Three Sirens: A Self-Learning Robotic Rock Band. http://www.the-three-sirens.info. Accessed July 2006.Google Scholar
- Bamberger, J. Music, Math, and Science: Towards an Integrated Curriculum. Journal for Learning Through Music, (Spring 2000), 32--35.Google Scholar
- Chordia, P. 2004. Automatic Tabla Transcription. PhD Thesis Stanford University.Google Scholar
- Coren, S. and Ward, L.M. Sensation and Perception. Harcourt Brace Jovanovich, San Diego, 1984.Google Scholar
- Coyle, E.J. and Shmulevich, I.A. System for Machine Recognition of Music Patterns. In Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing. Seattle, WA ,1998, 3597--3600.Google ScholarCross Ref
- Cycling74. Max/MSP. http://www.cycling74.com Accessed July 2006.Google Scholar
- Dannenberg, R. An On-line Algorithm for Real-Time Accompaniment. In Proceedings of the International Computer Music Conference Paris, France, 193--198.Google Scholar
- Dannenberg, R.B., Brown, B., Zeglin, G., and Lupish, R. McBlare - A Robotic Bagpipe Player. In Proceedings of the International Conference on New Interface for Musical Expression. Vancouver, Canada, 2005. 80--84. Google ScholarDigital Library
- Desain, P. and Honing, H.J. Rhythmic stability as explanation of category size. In Proceedings of the International Conference on Music Perception and Cognition, Sydney, 2002 CD-Rom.Google Scholar
- Foote, J. and Uchihashi, S. The Beat Spectrum: a new approach to rhythmic analysis. In Proceedings of the International Conference on Multimedia and Expo, Tokyo, Japan, 2001, 881--884.Google ScholarCross Ref
- Hewlett, B. and Selfreidge-Field, E. Melodic Similarity - Concepts, Procedures, and Applications. MIT Press. Cambridge, MA 1998.Google Scholar
- Jordà, S. Afasia: The Ultimate Homeric One-man multimedia-band. In Proceedings of the The International Conference on New Interfaces for Musical Expression. Dublin, Ireland, May 24-26 2002, 1--6. Google ScholarDigital Library
- Kapur, A. A History of Robotic Musical Instruments. In Proceedings of the 2005 International Computer Music Conference, Barcelona, Spain, 21--28.Google Scholar
- Lerdahl, F. and Jackendoff, R. A Generative Theory of Tonal Music. MIT Press, Cambridge, MA, 1983.Google Scholar
- Lewis, G. Too Many Notes: Computers, Complexity and Culture in Voyager Leonardo Music Journal 10 Cambridge, MA, (2000) 33--39.Google Scholar
- MakingThings. Teleo. http://www.makingthings.com. Accessed July 2006.Google Scholar
- Narmour, E. The Analysis and Cognition of Basic Melodic Structures. Chicago University of Chicago Press, 1990.Google Scholar
- Narmour, E. The Analysis and Cognition of Melodic Complexity: The Implication-Realization Model. Chicago University of Chicago Pres, 1992.Google Scholar
- Orio, N., Lemouton, S., Schwarz, D. Score Following: State of the Art and New Developments. In Proceedings of the International Conference on New Interfaces for Musical Expression. Montreal, Canada, 1992, 36--41. Google ScholarDigital Library
- Pachet, F. The continuator: Musical interaction with style. In Proceedings of the International Computer Music Conference. Göteborg, Sweden, 2002, 211--218.Google Scholar
- Papert, S. Mindstorms. New York Basic Books, 1980.Google Scholar
- Paulus, J. and Kalpuri, A. Measuring the Similarity of Rhythmic Patterns. In Proceedings of the 2002 International Conference on Music Information Retrieval. Paris, France, 150--156.Google Scholar
- Puckette, M. Real-time Audio Analysis Tools for Pd and MSP. In Proceedings of the International Computer Music Conference, Ann Arbor, MI, 1998, 109--112.Google Scholar
- Rae, G.W. Robotic Instruments. http://logosfoundation.org/instrum_gwr/automatons.html Accessed July 2006Google Scholar
- Rauscher, F.H, Shaw, G.L, and Ky, C.N. 1993. Music and spatial task performance Nature 365, 611Google Scholar
- Resnick, M., Martin, F., Sargent, R., and Silverman, B. 1996. Programmable Bricks: Toys to Think With. IBM Systems Journal, 35:3&4. Google ScholarDigital Library
- Rowe, R. 2004. Machine Musicianship. Cambridge, MA MIT Press. Google ScholarDigital Library
- Scheirer, E. 1998. Tempo and beat analysis of acoustic musical signals Journal of the Acoustical Society of America, 103:1 588--601.Google Scholar
- Schillinger, J. The Mathematical Basis of the Arts. Da Capo Press, New York, 1976.Google Scholar
- Singer, E., Feddersen, J., C., R., and Bowen, B. LEMUR's Musical Robots. In Proceedings of the International Conference on New Interfaces for Musical Expression. Hamamatsu, Japan, 2004. Google ScholarDigital Library
- Singer, E., Larke, K., and Bianciardi, D. LEMUR GuitarBot: MIDI Robotic String Instrument. In Proceedings of the International Conference on New Interfaces for Musical Expression. Montreal, Canada, 2003. Google ScholarDigital Library
- Sony. 2003. QRIO conductor Robot. http://www.sony.net/SonyInfo/QRIO/works/20040325e.html Accessed July 2007Google Scholar
- Takanishi, A. and Maeda, M. Development of Anthropomorphic Flutist Robot WF-3RIV. In the Proceedings of International Computer Music Conference, 1998, 328--331.Google Scholar
- Tanguiane, A. Artificial Perception and Music Recognition. Springer-Verlag, New York, 1993. Google ScholarDigital Library
- Taylor, M. Remembering How to Drum: Djembe Technique (DVD). Blue 7 Media, 2004.Google Scholar
- Tindale, A.R., Kapur, A., Schloss, W.A., and Tzanetakis, G. Indirect Acquisition of Percussion Gestures Using Timbre Recognition. In Proceedings of the Colloque Interdisciplinaire de Musicologie. Montreal, 2005.Google Scholar
- Toyota. 2004. Trumpet Robot. http://www.toyota.co.jp/en/special/robot Accessed July 2006.Google Scholar
- Trehub, S.E., Bull, D., and Thorpe, L.A. 1984. Infants' perception of melodies: The role of melodic contour Child Development, 55:1 821--830.Google Scholar
- Vercoe, B. The Synthetic Performer in the Context of Live Performance. In Proceedings of the International Computer Music Conference. Paris, France, 1984 199--200.Google Scholar
- Weinberg, G. 2005. Interconnected Musical Networks - Toward a Theoretical Framework, Computer Music Journal 29:2, MIT Press, Cambridge, MA, 23--39. Google ScholarDigital Library
- Weinberg, G., Lackner, T., and Jay, J. The Musical Fireflies - Learning About Mathematical Patterns in Music Through Expression and Play. In Proceedings of the Colloquium on Musical Informatics 2001. A'quila, Italy 146--149.Google Scholar
- Weinberg, G., Driscoll S., and Parry, M. Musical Interactions with a Perceptual Robotic Percussionist. In Proceedings of the of IEEE International Workshop on Robot and Human Interactive Communication, 2005, Nashville, TN 456--461.Google ScholarCross Ref
- Weinberg, G., Driscoll, S., and Thatcher, T. Jam'aa for Haile. In Proceedings of the International Conference and Exhibition on Computer Graphics and Interactive Techniques (SIGGRPAPH 2006). 2006 Boston MA. p. 124. Google ScholarDigital Library
- Winkler, T. Composing Interactive Music: Techniques and Ideas Using Max. Cambridge, MA MIT Press, 2001. Google ScholarDigital Library
Index Terms
- The interactive robotic percussionist: new developments in form, mechanics, perception and interaction design
Recommendations
Robot-human interaction with an anthropomorphic percussionist
CHI '06: Proceedings of the SIGCHI Conference on Human Factors in Computing SystemsThe paper presents our approach for human-machine interaction with an anthropomorphic mechanical percussionist that can listen to live players, analyze perceptual musical aspects in real-time, and use the product of this analysis to play along in a ...
Exploring percussive gesture on iPads with ensemble metatone
CHI '14: Proceedings of the SIGCHI Conference on Human Factors in Computing SystemsPercussionists are unique among western classical instrumentalists in that their artistic practice is defined by an approach to interaction rather than their instruments. While percussionists are accustomed to exploring non-traditional objects to create ...
Sonic Gestures Applied to a Percussive Dialogue in TanGram Using Wii Remotes
ICEC '09: Proceedings of the 8th International Conference on Entertainment ComputingTanGram, is an original music score composed by Carlos D. Perales for Percussion Quartet and Nintendo Wii Remotes used as a wireless interface for Live Electronics. This paper examines how this composition explores exiting research in interaction and ...
Comments