Real-time background music monitoring based on content-based retrieval

ABSTRACT

In this paper, we describe music monitoring in TV broadcasting based on content-based retrieval. A part of audio signals is sequentially extracted from TV broadcasting as a retrieval key, and a music DB that stores a great number of musical pieces is retrieved by this key based on content-based retrieval, and a musical piece is identified sequentially. In this way, we are able to carry out music monitoring. There are three necessary requirements important for realization of the music monitoring. They are robustness against non-stationary noise, real-time processing of large-scale music DB retrieval, and high granularity of the retrieval key. As a method of realizing robustness against non-stationary noise, we propose a partially similar retrieval method which improves retrieval accuracy by using the moment in which no superfluous noise is produced during the existence of non-stationary noise. In order to realize real-time processing of large-scale music DB retrieval, we adopt a coarse-to-fine strategy, and propose a spectral peaks hashing method which performs high-speed refining by using hashing. To calculate a hash value in this hashing, frequency channel numbers of the spectral peaks are used. In order to realize high granularity of the retrieval key, it is necessary to solve the problem of retrieval accuracy degradation associated with heightening the granularity. To improve this accuracy, we propose a detection-by-continuity method which uses music continuity. Moreover, by using music continuity to correct the starting point and the terminal point of a musical piece in TV broadcasting, the retrieval accuracy is improved further. In order to evaluate the effectiveness of the proposed methods, we performed experiments using a music DB which stores over 28,000 musical pieces (over 1800 hours) and TV broadcasting audio signals containing music and background music (BGM). The granularity of the retrieval key was set at about 0.5 seconds. Through these experiments, We verified that music monitoring was possible for over 90% of the total time of music and BGM used in TV broadcasting, and that real-time processing was possible.

REFERENCES

• 1: L. Boney, A. H. Tewfik, and K. N. Harmdy. digital watermarks for audio signals. In Internatioal conference on multimedia computing and systems, pages 473--480, 1996.
• 2: P. Cano, E. Battle, T. Kallker, and J. Haitsma. A review of algorithms for audio fingerprinting. In internatioanl Workshop on Multimedia Signal Processing, 2002.
• 3: P. Cano, E. Battle, H. Mayer, and H. Neuschmied. Robust sound modeling for song detection in broadcast audio. In Proc. AES 112th Int. Conv., 2002.
• 4: Jonathan Foote, An overview of audio information retrieval, Multimedia Systems, v.7 n.1, p.2-10, Jan. 1999 [doi>10.1007/s005300050106 ]
• 5: J. T. Foote. Content-based retrieval of music and audio. In Multimedia Storage and Archiving System II, Proc. of SPIE, volume 3229, pages 138--147, 1997.
• 6: E. Gómez, P. Cano, L. de~C. T. Gomes, and E. B. M. Bonnet. Mixed watermarking-fingeprinting approach for integrity verification of audio recordings. In Proc. of IEEE international telecommunications symposiom, 2002.
• 7: J. Haitsma and T. Kalker. A highly robust audio fingerprinting system. In Proc. of IMSIR 2002, October 2002.
• 8: K. Kashino, G. Smith, and H. Murase. Time-series active search for quick retrieval of audio and video. In Proc. of 1999 International Conference of Acoustics, Speech and Signal Processing, volume VI, pages 2993--2996, 1999.
• 9: D. Kirovski and H. Malvar. Robust spread-spectrum audio watermarking. In Proc. 2001 IEEE International Conference on Acoustics, Speech and Signal Processing, volume~3, pages 1345--1348, May 2001.
• 10: Darko Kirovski , Henrique Malvar , Yacov Yacobi, Multimedia content screening using a dual watermarking and fingerprinting system, Proceedings of the tenth ACM international conference on Multimedia, December 01-06, 2002, Juan-les-Pins, France [doi>10.1145/641007.641086]
• 11: B. Logan. Mel frequency cepstral coefficients for music modeling. In the first international symposium on music information retrieval, 2002.
• 12: H. Nagano, K. Kashino, and H. Murase. A fast search algorithm for background music signals based on search for numerous small signal components. In Proc. of 2003 IEEE International Conference on Acoustics, Speech and Signal Processing, volume V, pages 796--799, April 2003.
• 13: F. A. P. Petitcolas, R. J. Anderson, and M. G. Kuhn. Information hiding - a survey. In Proc. of IEEE special issue on Protection of multimedia content, pages 1062--1078, 1999.
• 14: Lawrence Rabiner , Biing-Hwang Juang, Fundamentals of speech recognition, Prentice-Hall, Inc., Upper Saddle River, NJ, 1993
• 15: A. Wang. An industrial strength audio search algorithm. In Online Proceedings of ISMIR2003. http://ismir2003.ismir.net/presentations/Wang.PDF.
• 16: Erling Wold , Thom Blum , Douglas Keislar , James Wheaton, Content-Based Classification, Search, and Retrieval of Audio, IEEE MultiMedia, v.3 n.3, p.27-36, September 1996 [doi>10.1109/93.556537 ]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.2 DATABASE MANAGEMENT
      H.2.4 Systems
          Subjects: Multimedia databases

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Search process

  I. Computing Methodologies
  I.5 PATTERN RECOGNITION
      I.5.4 Applications
          Subjects: Waveform analysis; Signal processing

General Terms:
Design, Performance

Keywords:
content-based retrieval, hashing, monitoring, music continuity, non-stationary noise, spectral peaks