Designing time-based interactions with multimedia

ABSTRACT

The current model of time in multimedia frameworks poses particular problems when designing multimedia systems with time-based interaction. We propose to expand and extend an existing distinction between semantic time and real time from music and film theory to multimedia systems design. The semantic time concept also forms the foundation of a new software framework for multimedia systems that we are building, that, unlike most existing frameworks, includes mechanisms for time-based effects such as time-stretching.

REFERENCES

• 1: J. Borchers, E. Lee, W. Samminger, and M. Mühlhäuser. Personal orchestra: A real-time audio/video system for interactive conducting. ACM Multimedia Systems Journal Special Issue on Multimedia Software Engineering, 9(5):458--465, March 2004. Errata published in next issue.
• 2: D. Bordwell and K. Thompson. Film Art: An Introduction. McGraw-Hill, New York, 2003.
• 3: David Jaffe, Ensemble timing in computer music, Computer Music Journal, v.9 n.4, p.38-48, Winter 1985 [doi>10.2307/3679621]
• 4: Eric Lee , Thorsten Karrer , Jan Borchers, Toward a Framework for Interactive Systems to Conduct Digital Audio and Video Streams, Computer Music Journal, v.30 n.1, p.21-36, March 2006 [doi>10.1162/014892606776021317]
• 5: Eric Lee , Teresa Marrin Nakra , Jan Borchers, You're the conductor: a realistic interactive conducting system for children, Proceedings of the 2004 conference on New interfaces for musical expression, p.68-73, June 03-05, 2004, Hamamatsu, Shizuoka, Japan

INDEX TERMS

Primary Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
        D.2.11 Software Architectures
            Subjects: Patterns (e.g., client/server, pipeline, blackboard)

Additional Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.2 Design Tools and Techniques
          Subjects: Software libraries; User interfaces

General Terms:
Design, Theory

Keywords:
multimedia frameworks, multimedia systems, time design, time-based media