Toward an understanding of similarity judgments for music digital library evaluation

ABSTRACT

This paper presents an analysis of 7,602 similarity judgments collected for the Symbolic Melodic Similarity (SMS) and Audio Music Similarity and Retrieval (AMS) evaluation tasks in the 2006 Music Information Retrieval Evaluation eXchange (MIREX). We discuss the influence of task definitions, as well as evaluation metrics on user perceptions of music similarity, and provide recommendations for future Music Digital Library/Music Information Retrieval research pertaining to music similarity.

REFERENCES

• 1: Downie, J. S., West, K., Ehmann, A., and Vincent, E. The 2005 Music Information Retrieval Evaluation eXchange (MIREX 2005): Preliminary overview. In Proceedings of the 6th International Conference on Music Information Retrieval (ISMIR 2005), Queen Mary, UK, 2005, 320--323.
• 2: Vincenzo Della Mea , Stefano Mizzaro, Measuring retrieval effectiveness: a new proposal and a first experimental validation, Journal of the American Society for Information Science and Technology, v.55 n.6, p.530-543, April 2004 [doi>10.1002/asi.10408]
• 3: MIREX Wiki. Available at: http://music-ir.org/mirexwiki/.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.7 Digital Libraries
          Subjects: User issues

General Terms:
Human Factors, Measurement, Performance

Keywords:
Evalutron 6000, MIREX, evaluation, music digital libraries, music information retrieval, music similarity