CaMus2: collaborative music performance with mobile camera phones

ABSTRACT

CaMus2 is a multi-user multi-phone extension of the CaMus system. Mobile camera phones use their cameras to track position, rotation, height, and other parameters over a marker sheet to allow interactive performance of music. Multiple camera phones can use the same or separate marker sheets and send their interaction parameters via Bluetooth to a computer where the sensor information is converted to MIDI format to allow control of a wide range of sound generation and manipulation hardware and software. The semantics of the mapping of MIDI message to performance parameters of the camera interactions are fed back into the visualization on the camera phone.

REFERENCES

• 1: F. Behrendt. Handymusik. Klangkunst und 'mobile devices'. Epos, 2005. Available online at: www.epos.uos.de/music/templates/buch.php?id=57 (retrieved on April 1, 2007).
• 2: William Carter , Leslie S. Liu, Location33: a mobile musical, Proceedings of the 2005 conference on New interfaces for musical expression, May 26-28, 2005, Vancouver, Canada
• 3: G. Essl and M. Rohs. Mobile STK for Symbian OS. In Proc. International Computer Music Conference, pages l278--281, New Orleans, November 2006.
• 4: George W. Fitzmaurice, Situated information spaces and spatially aware palmtop computers, Communications of the ACM, v.36 n.7, p.39-49, July 1993 [doi>10.1145/159544.159566]
• 5: Lalya Gaye , Lars Erik Holmquist , Frauke Behrendt , Atau Tanaka, Mobile music technology: report on an emerging community, Proceedings of the 2006 conference on New interfaces for musical expression, June 04-08, 2006, Paris, France
• 6: Lalya Gaye , Ramia Mazé , Lars Erik Holmquist, Sonic City: the urban environment as a musical interface, Proceedings of the 2003 conference on New interfaces for musical expression, May 22-24, 2003, Montreal, Quebec, Canada
• 7: M. Kaltenbrunner. Interactive music for mobile digital music players. In Inspirational Idea for the International Computer Music Conference (ICMC), Barcelona, Spain, Sept. 2005.
• 8: G. Levin. Dialtones - a telesymphony. www.flong.com/telesymphony, September 2 2001. Retrieved on April 1, 2007.
• 9: Sumit Mehra , Peter Werkhoven , Marcel Worring, Navigating on handheld displays: Dynamic versus static peephole navigation, ACM Transactions on Computer-Human Interaction (TOCHI), v.13 n.4, p.448-457, December 2006 [doi>10.1145/1188816.1188818]
• 10: Eduardo Reck Miranda , Marcelo Wanderley, New Digital Musical Instruments: Control And Interaction Beyond the Keyboard (Computer Music and Digital Audio Series), A-R Editions, Inc., Madison, WI, 2006
• 11: Michael Rohs , Georg Essl, Which one is better?: information navigation techniques for spatially aware handheld displays, Proceedings of the 8th international conference on Multimodal interfaces, November 02-04, 2006, Banff, Alberta, Canada [doi>10.1145/1180995.1181016]
• 12: Michael Rohs , Georg Essl , Martin Roth, CaMus: live music performance using camera phones and visual grid tracking, Proceedings of the 2006 conference on New interfaces for musical expression, June 04-08, 2006, Paris, France
• 13: Greg Schiemer , Mark Havryliv, Pocket Gamelan: tuneable trajectories for flying sources in Mandala 3 and Mandala 4, Proceedings of the 2006 conference on New interfaces for musical expression, June 04-08, 2006, Paris, France
• 14: Steven Strachan , Parisa Eslambolchilar , Roderick Murray-Smith , Stephen Hughes , Sile O'Modhrain, GpsTunes: controlling navigation via audio feedback, Proceedings of the 7th international conference on Human computer interaction with mobile devices & services, September 19-22, 2005, Salzburg, Austria [doi>10.1145/1085777.1085831]
• 15: Atau Tanaka , Petra Gemeinboeck, A framework for spatial interaction in locative media, Proceedings of the 2006 conference on New interfaces for musical expression, June 04-08, 2006, Paris, France
• 16: Nigel Warren , Matt Jones , Steve Jones , David Bainbridge, Navigation via continuously adapted music, CHI '05 extended abstracts on Human factors in computing systems, April 02-07, 2005, Portland, OR, USA [doi>10.1145/1056808.1057038]
• 17: D. Waters. Mobile music challenges 'iPod age'. BBC News Online, Monday, 7 March 2005. Available online at news.bbc.co.uk/1/hi/technology/4315481.stm (retrieved on April 1, 2007).
• 18: T. Yamauchi and I. Toru. Mobile user-interface for music (poster). In Proceedings of the Mobile Music Workshop, Vancouver, Canada, May 25 2005.
• 19: Ka-Ping Yee, Peephole displays: pen interaction on spatially aware handheld computers, Proceedings of the SIGCHI conference on Human factors in computing systems, April 05-10, 2003, Ft. Lauderdale, Florida, USA [doi>10.1145/642611.642613]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Input devices and strategies (e.g., mouse, touchscreen)

Additional Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.1 Multimedia Information Systems
          Subjects: Audio input/output
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Interaction styles (e.g., commands, menus, forms, direct manipulation); Screen design (e.g., text, graphics, color)
      H.5.3 Group and Organization Interfaces
          Subjects: Synchronous interaction; Collaborative computing

General Terms:
Design, Human Factors

Keywords:
camera phones, camera-based interaction, collaboration, mobile phones, music performance, small displays, spatially aware displays