SPREAD: sound propagation and perception for autonomous agents in dynamic environments

ABSTRACT

The perception of sensory information and its impact on behavior is a fundamental component of being human. While visual perception is considered for navigation, collision, and behavior selection, the acoustic domain is relatively unexplored. Recent work in acoustics focuses on synthesizing sound in 3D environments; however, the perception of acoustic signals by a virtual agent is a useful and realistic adjunct to any behavior selection mechanism. In this paper, we present SPREAD, a novel agent-based sound perception model using a discretized sound packet representation with acoustic features including amplitude, frequency range, and duration. SPREAD simulates how sound packets are propagated, attenuated, and degraded as they traverse the virtual environment. Agents perceive and classify the sounds based on the locally-received packet set using a hierarchical clustering scheme, and have individualized hearing and understanding of their surroundings. Using this model, we demonstrate several simulations that greatly enrich controls and outcomes.

References

1. Bee, M., and Micheyl, C. 2008. The cocktail party problem: What is it? how can it be solved? and why should animal behaviorists study it? J. of Comparative Psychology 122, 3, 235.
2. Bonebright, T. 2001. Perceptual structure of everyday sounds: A multidimensional scaling approach. In Proc. of the 7th international conference on auditory display, 73--78.
3. Chandak, A., Lauterbach, C., Taylor, M., Ren, Z., and Manocha, D. 2008. Ad-frustum: Adaptive frustum tracing for interactive sound propagation. IEEE TVCG 14, 6.
4. Cony, C., de Lima Bicho, A., Jung, C., Magalhaes, L., and Musse, S. 2007. A perceptive model for virtual agents in crowds. In CGI, vol. 1, 141--150.
5. Cowling, M., and Sitte, R. 2003. Comparison of techniques for environmental sound recognition. Pattern Recognition Letters 24, 15, 2895--2907.
6. Dekel, O., Keshet, J., and Singer, Y. 2005. An online algorithm for hierarchical phoneme classification. In MLMI, 146--158.
7. Funkhouser, T., Carlbom, I., Elko, G., Pingali, G., Sondhi, M., and West, J. 1998. A beam tracing approach to acoustic modeling for interactive virtual environments. In SIGGRAPH, ACM, 21--32.
8. Gygi, B., Kidd, G., and Watson, C. 2007. Similarity and categorization of environmental sounds. Attention, Perception, & Psychophysics 69, 6, 839--855.
9. Herrero, P., and de Antonio, A. 2003. Introducing human-like hearing perception in intelligent virtual agents. In AAMAS, ACM, 733--740.
10. Holland, J., Dabelsteen, T., Pedersen, S., and Larsen, O. 1998. Degradation of wren troglodytes troglodytes song: implications for information transfer and ranging. J. of the Acoustical Society of America 103, 2154.
11. Hory, C., Martin, N., and Chehikian, A. 2002. Spectrogram segmentation by means of statistical features for non-stationary signal interpretation. Signal Processing, IEEE Transactions on 50, 12, 2915--2925.
12. James, D., Barbič, J., and Pai, D. 2006. Precomputed acoustic transfer: output-sensitive, accurate sound generation for geometrically complex vibration sources. In ACM TOG.
13. Kagawa, Y., Tsuchiya, T., Fujii, B., and Fujioka, K. 1998. Discrete huygens'model approach to sound wave propagation. J. of Sound and Vibration 218, 3, 419--444.
14. Kapadia, M., Singh, S., Reinman, G., and Faloutsos, P. 2011. A Behavior-Authoring Framework for Multiactor Simulations. Computer Graphics & Applications, IEEE 31, 6, 45--55.
15. Kristiansen, U., and Viggen. 2010. Computational methods in acoustics. Compendium, NTNU.
16. Li, S., and Loew, M. 1987. Adjacency detection using quad-codes. Communications of the ACM 30, 7, 627--631.
17. Mast, T. 2000. Empirical relationships between acoustic parameters in human soft tissues. Acoustics Research Letters Online 1, 2, 37--42.
18. Monzani, J., and Thalmann, D. 2000. A sound propagation model for interagents communication. In Virtual Worlds, Springer, 135--146.
19. Ondřej, J., Pettré, J., Olivier, A., and Donikian, S. 2010. A synthetic-vision based steering approach for crowd simulation. ACM TOG 29, 4, 123.
20. O'Sullivan, C., and Ennis, C. 2011. Metropolis: multisensory simulation of a populated city. In Proc. Intl. Conf. on Games and Virtual Worlds for Serious Applications, IEEE Computer Society, 1--7.
21. Pelechano, N., Allbeck, J., and Badler, N. 2008. Virtual crowds: Methods, simulation, and control. Synthesis Lectures on Computer Graphics and Animation 3, 1, 1--176.
22. Raghuvanshi, N., Narain, R., and Lin, M. 2009. Efficient and accurate sound propagation using adaptive rectangular decomposition. IEEE TVCG 15, 5, 789--801.
23. Raghuvanshi, N., Snyder, J., Mehra, R., Lin, M., and Govindaraju, N. 2010. Precomputed wave simulation for real-time sound propagation of dynamic sources in complex scenes. ACM Transactions on Graphics (TOG) 29, 4, 68.
24. Savioja, L., Huopaniemi, J., Lokki, T., and Vaananen, R. 1999. Creating interactive virtual acoustic environments. J. of the Audio.
25. Shoulson, A., Marshak, N., Kapadia, M., and Badler, N. I. 2013. ADAPT: the agent development and prototyping testbed. In ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games, I3D, 9--18.
26. Takala, T., and Hahn, J. 1992. Sound rendering. In ACM SIGGRAPH Computer Graphics, vol. 26, ACM, 211--220.
27. Thalmann, D. 2007. Crowd simulation. Wiley Online Library.
28. Turetsky, R., and Ellis, D. 2003. Ground-truth transcriptions of real music from force-aligned midi syntheses. ISMIR 2003, 135--141.
29. Unity3D. 2012. Unity3d game engine. http://unity3d.com.
30. Xu, C., Maddage, N. C., and Shao, X. 2005. Automatic music classification and summarization. Speech and Audio Processing, IEEE Transactions on 13, 3, 441--450.

Index Terms

• Computing methodologies

  • Computer graphics

    • Animation