Motion swarms: video interaction for art in complex environments

ABSTRACT

We create interactive art that can be enjoyed by groups such as audiences at public events with the intent to encourage communication with those around us as we play with the art. Video systems are an attractive mechanism to provide interaction with artwork. However, public spaces are complex environments for video analysis systems. Interaction becomes even more difficult when the art is viewed by large groups of people. We describe a video system for interaction with art in public spaces and with large audiences using a model-free, appearance-based approach. Our system extracts parameters that describe the field of motion seen by a camera, and then imposes structure on the scene by introducing a swarm of particles that moves in reaction to the motion field. Constraints placed on the particle movement impose further structure on the motion field. The artistic display reacts to the particles in a manner that is interesting and predictable for participants. We demonstrate our video interaction system with a series of interactive art installations tested with the assistance of a volunteer audience.

REFERENCES

• 1: Apple Computer Inc. Working with quartz composer. Retrieved May 30, 2006, from http://developer.apple.com/graphicsimaging/quartz/quartzcomposer.html, 2005.
• 2: A. M. Baumberg and D. C. Hogg. Learning flexible models from image sequences. Technical Report 93.36, University of Leeds School of Computer Studies, October 1993.
• 3: Adam Baumberg , David Hogg, Generating spatiotemporal models from examples, Proceedings of the 6th British conference on Machine vision (Vol. 2), p.413-422, July 1995, Birmingham, United Kingdom
• 4: A. Bobick , J. Davis, An Appearance-Based Representation of Action, Proceedings of the 1996 International Conference on Pattern Recognition (ICPR '96) Volume I, p.307, August 25-30, 1996
• 5: A. Bobick , J. Davis, Real-time recognition of activity using temporal templates, Proceedings of the 3rd IEEE Workshop on Applications of Computer Vision (WACV '96), p.39, December 02-04, 1996
• 6: J. E. Boyd. Video phase-locked loops in gait recognition. In International Conference on Computer Vision, pages 696--703, Vancouver, BC, 2001.
• 7: Jeffrey E. Boyd , Gerald Hushlak , Christian J. Jacob, SwarmArt: interactive art from swarm intelligence, Proceedings of the 12th annual ACM international conference on Multimedia, October 10-16, 2004, New York, NY, USA [doi>10.1145/1027527.1027674]
• 8: Cynthia Breazeal , Andrew Brooks , Jesse Gray , Matt Hancher , John McBean , Dan Stiehl , Joshua Strickon, Interactive robot theatre, Communications of the ACM, v.46 n.7, July 2003 [doi>10.1145/792704.792733]
• 9: C. Bregler , J. Malik, Tracking People with Twists and Exponential Maps, Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, p.8, June 23-25, 1998
• 10: R. Cutler and L. Davis. Robust periodic motion and motion symmetry detection. In Computer Vision and Pattern Recognition 2000, 2000.
• 11: R. Deriche, Fast Algorithms for Low-Level Vision, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.12 n.1, p.78-87, January 1990 [doi>10.1109/34.41386 ]
• 12: eInstruction. Classroom performance system. Retrieved May 30, 2006, from http://www.einstruction.com/.
• 13: H. Fujiyoshi and A. J. Lipton. Real-time human motion analysis by image skeletonization. In DARPA Image Understanding Workshop, Monterey, California, November 1998.
• 14: I. Haritaoglu, D. Harwood, and L. S. Davis. Ghost: a human body part labelling system using silhouettes. In DARPA Image Understanding Workshop, pages 229--235, Monterey, California, November 1998.
• 15: E. A. Hunter P. H. Kelly R. C. Jain, Estimation of Articulated Motion Using Kinematically Constrained Mixture Densities, Proceedings of the 1997 IEEE Workshop on Motion of Non-Rigid and Articulated Objects (NAM '97), p.10, June 16-July 16, 1997
• 16: G. Johansson. Visual motion perception. Scientific American, pages 76--88, June 1975.
• 17: J. J. Little and J. E. Boyd. Recognizing people by their gait: the shape of motion. Videre, 1(2):1--32, 1998.
• 18: Fang Liu , Rosalind W. Picard, Finding Periodicity in Space and Time, Proceedings of the Sixth International Conference on Computer Vision, p.376, January 04-07, 1998
• 19: Techniques for Interactive Audience Participation, Proceedings of the 4th IEEE International Conference on Multimodal Interfaces, p.15, October 14-16, 2002
• 20: M. McLuhan and Q. Fiore. The Medium is the Message. Bantam Books/Random House, 1967.
• 21: Ramprasad Polana , Randal C. Nelson, Detection and Recognition of Periodic, Nonrigid Motion, International Journal of Computer Vision, v.23 n.3, p.261-282, June/July 1997 [doi>10.1023/A:1007975200487 ]
• 22: Henry A. Rowley , James M. Rehg, Analyzing Articulated Motion Using Expectation-Maximization, Proceedings of the 1997 Conference on Computer Vision and Pattern Recognition (CVPR '97), p.935, June 17-19, 1997
• 23: Jürgen Scheible , Timo Ojala, MobiLenin combining a multi-track music video, personal mobile phones and a public display into multi-user interactive entertainment, Proceedings of the 13th annual ACM international conference on Multimedia, November 06-11, 2005, Hilton, Singapore [doi>10.1145/1101149.1101178]
• 24: L. Spector, J. Klein, C. Perry, and M. Feinstein. Emergence of collective behavior in evolving populations of flying agents. In E. C.-P. e. al., editor, Genetic and Evolutionary Computation Conference (GECCO-2003), pages 61--73, Chicago, IL, 2003. Springer-Verlag.
• 25: K. Vonnegut. A Man Without a Country. Seven Stories Press, 2005.
• 26: S. Wachtert , H.-H. Nageltt, Tracking of Persons in Monocular Image Sequences, Proceedings of the 1997 IEEE Workshop on Motion of Non-Rigid and Articulated Objects (NAM '97), p.2, June 16-July 16, 1997
• 27: Christopher Richard Wren , Ali Azarbayejani , Trevor Darrell , Alex Paul Pentland, Pfinder: Real-Time Tracking of the Human Body, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.19 n.7, p.780-785, July 1997 [doi>10.1109/34.598236 ]

INDEX TERMS

Primary Classification:
  J. Computer Applications
  J.5 ARTS AND HUMANITIES
      Subjects: Fine arts

Additional Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.1 Multimedia Information Systems
          Subjects: Animations; Video (e.g., tape, disk, DVI)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Input devices and strategies (e.g., mouse, touchscreen)

  I. Computing Methodologies
  I.4 IMAGE PROCESSING AND COMPUTER VISION
      I.4.8 Scene Analysis
          Subjects: Motion; Tracking

General Terms:
Algorithms, Design, Experimentation, Human Factors

Keywords:
art installation, audience interaction, interaction through video, motion analysis, motion history image, swarm art