Relevance information: a loss of entropy but a gain for IDF?

ABSTRACT

When investigating alternative estimates for term discriminativeness, we discovered that relevance information and idf are much closer related than formulated in classical literature. Therefore, we revisited the justification of idf as it follows from the binary independent retrieval (BIR) model. The main result is a formal framework uncovering the close relationship of a generalised idf and the BIR model. The framework makes explicit how to incorporate relevance information into any retrieval function that involves an idf-component.In addition to the idf-based formulation of the BIR model, we propose Poisson-based estimates as an alternative to the classical estimates, this being motivated by the superiority of Poisson-based estimates for the within-document term frequencies. The main experimental finding is that a Poisson-based idf is superior to the classical idf, where the superiority is particularly evident for long queries.

REFERENCES

• 1: G. Amati. Probability Models for Information Retrieval based on Divergence from Randomness. PhD thesis, Glasgow University, June 2003.
• 2: K.W. Church and W.A. Gale. Inverse document frequency: A measure of deviations from poisson. In Third Workshop on Very Large Corpora, ACL Anthology, 1995.
• 3: A.P. de Vries and D. Hiemstra. The Mirror DBMS at TREC-8. In Proceedings of the Eighth Text Retrieval Conference TREC-8, pages 725--734, Gaithersburg, Maryland, November 1999.
• 4: Djoerd Hiemstra , Stephen Robertson , Hugo Zaragoza, Parsimonious language models for information retrieval, Proceedings of the 27th annual international ACM SIGIR conference on Research and development in information retrieval, July 25-29, 2004, Sheffield, United Kingdom [doi>10.1145/1008992.1009025]
• 5: J. Lafferty and Ch. Zhai. Probabilistic Relevance Models Based on Document and Query Generation, chapter 1. Kluwer, 2002.
• 6: S. E. Robertson , S. Walker, Some simple effective approximations to the 2-Poisson model for probabilistic weighted retrieval, Proceedings of the 17th annual international ACM SIGIR conference on Research and development in information retrieval, p.232-241, July 03-06, 1994, Dublin, Ireland
• 7: S. E. Robertson , S. Walker , M. M. Hancock-Beaulieu, Large test collection experiments on an operational, interactive system: Okapi at TREC, Information Processing and Management: an International Journal, v.31 n.3, p.345-360, May-June 1995
• 8: S.E. Robertson and K. Sparck Jones. Relevance weighting of search terms. Journal of the American Society for Information Science, 27:129--146, 1976.
• 9: Stephen Robertson. Understanding inverse document frequency: on theoretical arguments. Journal of Documentation, 60(5):503--520, 2004.
• 10: Thomas Roelleke, A frequency-based and a poisson-based definition of the probability of being informative, Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval, July 28-August 01, 2003, Toronto, Canada [doi>10.1145/860435.860478]
• 11: T. Roelleke, T. Tsikrika, and G. Kazai. A general matrix framework for modelling information retrieval. Journal on Information Processing & Management (IP&M), 2005. To appear.
• 12: Ian Ruthven , Mounia Lalmas, A survey on the use of relevance feedback for information access systems, The Knowledge Engineering Review, v.18 n.2, p.95-145, June 2003 [doi>10.1017/S0269888903000638]
• 13: G. Salton , C. Buckley, On the use of spreading activation methods in automatic information, Proceedings of the 11th annual international ACM SIGIR conference on Research and development in information retrieval, p.147-160, May 1988, Grenoble, France [doi>10.1145/62437.62447]
• 14: C. J. Van Rijsbergen, Information Retrieval, Butterworth-Heinemann, Newton, MA, 1979
• 15: S. K. M. Wong , Y. Y. Yao, On modeling information retrieval with probabilistic inference, ACM Transactions on Information Systems (TOIS), v.13 n.1, p.38-68, Jan. 1995 [doi>10.1145/195705.195713]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval

General Terms:
Experimentation, Theory

Keywords:
binary independent retrieval model, information retrieval, inverse document frequency, probability of relevance, relevance feedback