A statistical approach to retrieval under user-dependent uncertainty in query-by-humming systems

ABSTRACT

Robustly addressing uncertainty in query formulation and search is one of the most challenging problems in multimedia information retrieval (MIR) systems. In this paper, a statistical approach to the problem of retrieval under the effect of uncertainty in Query by Humming (QBH) systems is presented. Direct transcription of audio to pitch and duration symbols is performed. From the transcribed data vector, finger prints that carry a fixed length of information from characteristic local points of the hummed melody are extracted. Instead of employing the humming input as a whole, extracted characteristic information packages are used for search through the database. The distance for each finger print to the original melodies in the database is calculated and converted to probabilistic similarity measures. Melodies with the highest similarity measures are returned to the user as the most likely query result. This algorithm is tested with manually annotated data comprising 250 humming samples in conjunction with a database of 200 pre-processed midi files. Retrieval accuracy of 94 percent is demonstrated for the samples of subjects that have some musical training/background compared to 72 percent accuracy achieved for the samples of non-trained subjects. Results also show that extracting finger prints with respect to characteristic local points of the hummed tune is an effective and robust way for search and retrieval under the effect of uncertainty

REFERENCES

• 1: Shih H.-H., Narayanan, S. S. and Kuo, C.-C. J. An HMM-based approach to humming transcription. In Proceedings of IEEE International Conference on Multimedia and Expo (ICME2002), August 2002.
• 2: Shih H.-H., Narayanan, S. S. and Kuo, C.-C. J. Multidimensional Humming Transcription Using Hidden Markov Models for Query by Humming Systems. In Proceedings of IEEE International conference on Acoustics Speech and Signal Processing, 2003
• 3: Erdem Unal , S. S. Narayanan , H. H. Shih , Elaine Chew , C. C. Jay Kuo, Creating data resources for designing user-centric frontends for query by humming systems, Proceedings of the 5th ACM SIGMM international workshop on Multimedia information retrieval, November 07-07, 2003, Berkeley, California [doi>10.1145/973264.973284]
• 4: Bamberger, J. Turning Music Theory on its Ear. International Journal of Computers for Mathematical Learning Vol.1, No.1, 1996
• 5: Desain, P, Honing, H. The formation of rhythmic categories and metric priming. Music Perception, 2003, Vol 32, pp 341--365
• 6: Asif Ghias , Jonathan Logan , David Chamberlin , Brian C. Smith, Query by humming: musical information retrieval in an audio database, Proceedings of the third ACM international conference on Multimedia, p.231-236, November 05-09, 1995, San Francisco, California, United States [doi>10.1145/217279.215273]
• 7: Rodger J. McNab , Lloyd A. Smith , Ian H. Witten , Clare L. Henderson , Sally Jo Cunningham, Towards the digital music library: tune retrieval from acoustic input, Proceedings of the first ACM international conference on Digital libraries, p.11-18, March 20-23, 1996, Bethesda, Maryland, United States [doi>10.1145/226931.226934]
• 8: Rodger J. McNab , Lloyd A. Smith , Ian H. Witten , Clare L. Henderson, Tune Retrieval in the Multimedia Library, Multimedia Tools and Applications, v.10 n.2-3, p.113-132, April 2000
• 9: Steven Blackburn , David DeRoure, A tool for content based navigation of music, Proceedings of the sixth ACM international conference on Multimedia, p.361-368, September 13-16, 1998, Bristol, United Kingdom [doi>10.1145/290747.290802]
• 10: Pierre-Yves Rolland , Gailius Raškinis , Jean-Gabriel Ganascia, Musical content-based retrieval: an overview of the Melodiscov approach and system, Proceedings of the seventh ACM international conference on Multimedia (Part 1), p.81-84, October 30-November 05, 1999, Orlando, Florida, United States [doi>10.1145/319463.319473]
• 11: Shih, H.-H., Zhang, T. and Kuo, C.-C. J. Real-time retrieval of song from music database with query-by-humming. In Proceedings of ISMIP (1999), 251--57.
• 12: Chen B. and Roger Jang, J.-S. Query by Singing. In Proceedings of 11th IPPR Conference on Computer Vision, Graphics and Image Processing (Taiwan, 1998).
• 13: Lu, L., You, H., and Zhang, H.-J. A new approach to query by humming in music retrieval. In Proceedings of IEEE International Conference on Multimedia and Expo (2001)
• 14: Haus, G. and Pollstri, E. An Audio Front End for Query-by-Humming Systems. In Proceedings of ISMIR 2001(Bloomington, Indiana, October 2001)
• 15: Yunyue Zhu , Dennis Shasha, Warping indexes with envelope transforms for query by humming, Proceedings of the 2003 ACM SIGMOD international conference on Management of data, June 09-12, 2003, San Diego, California [doi>10.1145/872757.872780]
• 16: Huron, D. Tone and Voice: A Derivation of the Rules of Voice-leading from Perceptual Principles. Music Perception, Vol. 19, No. 1 (2001) pp. 1--64.
• 17: Rossing, T. D., Science of Sound, 3rd ed. (with F. Richard Moore, Paul A. Wheeler), Addison-Wesley, San Francisco, 2002
• 18: Capleton., B. Perfect Pitch http://www.amarilli.co.uk/piano/perfectp.asp

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Query formulation

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Retrieval models; Selection process; Search process

General Terms:
Algorithms, Design, Human Factors

Keywords:
query-by-humming, retrieval, uncertainty