Interactive sound synthesis for large scale environments

ABSTRACT

We present an interactive approach for generating realistic physically-based sounds from rigid-body dynamic simulations. We use spring-mass systems to model each object's local deformation and vibration, which we demonstrate to be an adequate approximation for capturing physical effects such as magnitude of impact forces, location of impact, and rolling sounds. No assumption is made about the mesh connectivity or topology. Surface meshes used for rigid-body dynamic simulation are utilized for sound simulation without any modifications. We use results in auditory perception and a novel priority-based quality scaling scheme to enable the system to meet variable, stringent time constraints in a real-time application, while ensuring minimal reduction in the perceived sound quality. With this approach, we have observed up to an order of magnitude speed-up compared to an implementation without the acceleration. As a result, we are able to simulate moderately complex simulations with upto hundreds of sounding objects at over 100 frames per second (FPS), making this technique well suited for interactive applications like games and virtual environments. Furthermore, we utilize OpenAL and EAX™ on Creative Sound Blaster Audigy 2™ cards for fast hardware-accelerated propagation modeling of the synthesized sound.

References

1. Chaigne, A., and Doutaut, V. 1997. Numerical simulations of xylophones. i. time domain modeling of the vibrating bars. J. Acoust. Soc. Am. 101, 1, 539--557.
2. Chung, J. Y., Liu, J., and Lin, K. J. 1987. Scheduling real-time, periodic jobs using imprecise results. In Proc. IEEE RTS.
3. Dongarra, J. J. 2005. Performance of various computers using standard linear equations software (linpack benchmark report). Tech. rep., Knoxville, TN, USA.
4. Florens, J. L., and Cadoz, C. 1991. The physical model: modeling and simulating the instrumental universe. In Representations of Musical Signals, G. D. Poli, A. Piccialli, and C. Roads, Eds. MIT Press, Cambridge, MA, USA, 227--268.
5. Fouad, H., Ballas, J., and Hahn, J. 1997. Perceptually based scheduling algorithms for real-time synthesis of complex sonic environments. In Proc. Int. Conf. Auditory Display.
6. Guendelman, E., Bridson, R., and Fedkiw, R. 2003. Nonconvex rigid bodies with stacking. ACM Trans. on Graphics (Proc. of ACM SIGGRAPH) 22, 871--878.
7. Kim, Y. J., Lin, M. C., and Manocha, D. 2002. DEEP: an incremental algorithm for penetration depth computation between convex polytopes. Proc. of IEEE Conference on Robotics and Automation, 921--926.
8. Mirtich, B., and Canny, J. 1995. Impulse-based simulation of rigid bodies. In 1995 Symposium on Interactive 3D Graphics, P. Hanrahan and J. Winget, Eds., ACM SIGGRAPH, 181--188. ISBN 0-89791-736-7.
9. O'Brien, J. F., Cook. P. R., and Essl, G. 2001. Synthesizing sounds from physically based motion. In SIGGRAPH '01: Proceedings of the 28th annual conference on Computer graphics and interactive techniques, ACM Press, New York, NY, USA, 529--536.
10. O'Brien, J. F., Shen, C., and Gatchalian. C. M. 2002. Synthesizing sounds from rigid-body simulations. In The ACM SIGGRAPH 2002 Symposium on Computer Animation, ACM Press, 175--181.
11. Sek, A., and Moore. B. C. 1995. Frequency discrimination as a function of frequency, measured in several ways. J. Acoust. Soc. Am. 97, 4 (April), 2479--2486.
12. Van Den Doel, K., and Pai, D. K. 1996. Synthesis of shape dependent sounds with physical modeling. In Proceedings of the International Conference on Auditory Displays.
13. Van Den Doel, K., and Pai, D. K. 1998. The sounds of physical shapes. Presence 7, 4, 382--395.
14. Van Den Doel, K., Kry, P. G., and Pai, D. K. 2001. Foleyautomatic: physically-based sound effects for interactive simulation and animation. In SIGGRAPH '01: Proceedings of the 28th annual conference on Computer graphics and interactive techniques, ACM Press, New York, NY, USA, 537--544.
15. Van Den Doel, K., Knott, D., and Pai, D. K. 2004. Interactive simulation of complex audiovisual scenes. Presence: Teleoper. Virtual Environ. 13, 1, 99--111.
16. Zwicker, E., and Fastl, H. 1990. In Psychoacoustics. Springer-Verlag, Berlin.

Index Terms

• Applied computing

  • Arts and humanities

    • Sound and music computing

• Computing methodologies

  • Computer graphics

    • Animation

      • Physical simulation

    • Shape modeling

• Information systems

  • Information retrieval

    • Specialized information retrieval

      • Multimedia and multimodal retrieval

        • Music retrieval

• Mathematics of computing

  • Continuous mathematics

    • Topology

      • Geometric topology