A parallel derivation of probabilistic information retrieval models

ABSTRACT

This paper investigates in a stringent athematical formalism the parallel derivation of three grand probabilistic retrieval models: binary independent retrieval (BIR), Poisson model (PM), and language modelling (LM).The investigation has been motivated by a number of questions. Firstly, though sharing the same origin, namely the probability of relevance, the models differ with respect to event spaces. How can this be captured in a consistent notation, and can we relate the event spaces? Secondly, BIR and PM are closely related, but how does LM fit in? Thirdly, how are tf-idf and probabilistic models related? .The parallel investigation of the models leads to a number of formalised results:

1. BIR and PM assume the collection to be a set of non-relevant documents, whereas LM assumes the collection to be a set of terms from relevant documents.
2. PM can be viewed as a bridge connecting BIR and LM.
3. A BIR-LM equivalence explains BIR as a special LM case.
4. PM explains tf-idf, and both, BIR and LM probabilities express tf-idf in a dual way.

.

REFERENCES

• 1: G. Amati. Probability Models for Information Retrieval based on Divergence from Randomness Ph thesis, Glasgow University, June 2003.
• 2: Gianni Amati , C. J. van Rijsbergen, Term Frequency Normalization via Pareto Distributions, Proceedings of the 24th BCS-IRSG European Colloquium on IR Research: Advances in Information Retrieval, p.183-192, March 25-27, 2002
• 3: Gianni Amati , Cornelis Joost Van Rijsbergen, Probabilistic models of information retrieval based on measuring the divergence from randomness, ACM Transactions on Information Systems (TOIS), v.20 n.4, p.357-389, October 2002 [doi>10.1145/582415.582416]
• 4: Marcia J. Bates. After the dot-bomb: Getting web information retrieval right this time. First Monday 7(7), 2002.
• 5: K. Church and W Gale. Inverse document frequency (idf): A measure of deviation from poisson.In Proceedings of the Third Workshop on Very Large Corpora pages 121--130, 1995.
• 6: W. Bruce Croft , John Lafferty, Language Modeling for Information Retrieval, Kluwer Academic Publishers, Norwell, MA, 2003
• 7: Arjen P. de Vries , Thomas Roelleke, Relevance information: a loss of entropy but a gain for IDF?, Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval, August 15-19, 2005, Salvador, Brazil [doi>10.1145/1076034.1076084]
• 8: joerd Hiemstra. A probabilistic justification for using tf.idf term weighting in information retrieval. International Journal on Digital Libraries 3(2): 131--139, 2000.
• 9: K. Sparck Jones, S. E. Robertson,. Hiemstra, and H. Zaragoza. Language modelling and relevance. Language Modelling for Information Retrieval pages 57--70,2003.
• 10: Wessel Kraaij , Thijs Westerveld , Djoerd Hiemstra, The Importance of Prior Probabilities for Entry Page Search, Proceedings of the 25th annual international ACM SIGIR conference on Research and development in information retrieval, August 11-15, 2002, Tampere, Finland [doi>10.1145/564376.564383]
• 11: John Lafferty and ChengXiang Zhai. Probabilistic Relevance Models Based on Document and Query Generation chapter 1. In Croft and Lafferty {6}, 2002.
• 12: Eugene L. Margulis, N-Poisson document modelling, Proceedings of the 15th annual international ACM SIGIR conference on Research and development in information retrieval, p.177-189, June 21-24, 1992, Copenhagen, Denmark [doi>10.1145/133160.133195]
• 13: Jay M. Ponte , W. Bruce Croft, A language modeling approach to information retrieval, Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval, p.275-281, August 24-28, 1998, Melbourne, Australia [doi>10.1145/290941.291008]
• 14: S. E. Robertson , S. Walker, Some simple effective approximations to the 2-Poisson model for probabilistic weighted retrieval, Proceedings of the 17th annual international ACM SIGIR conference on Research and development in information retrieval, p.232-241, July 03-06, 1994, Dublin, Ireland
• 15: S. E. Robertson , S. Walker , M. M. Hancock-Beaulieu, Large test collection experiments on an operational, interactive system: Okapi at TREC, Information Processing and Management: an International Journal, v.31 n.3, p.345-360, May-June 1995
• 16: S. E. Robertson. Understanding inverse document frequency:On theoretical arguments for idf. Journal of Documentation 60:503--520,2004.
• 17: S. E. Robertson and K. Sparck Jones. Relevance weighting of search terms.Journal of the American Society for Information Science 27: 129--146, 1976.
• 18: Stephen Robertson, On Event Spaces and Probabilistic Models in Information Retrieval, Information Retrieval, v.8 n.2, p.319-329, April 2005 [doi>10.1007/s10791-005-5665-9]
• 19: Thomas Rölleke , Theodora Tsikrika , Gabriella Kazai, A general matrix framework for modelling information retrieval, Information Processing and Management: an International Journal, v.42 n.1, p.4-30, January 2006 [doi>10.1016/j.ipm.2004.11.006]
• 20: Amit Singhal , Chris Buckley , Mandar Mitra, Pivoted document length normalization, Proceedings of the 19th annual international ACM SIGIR conference on Research and development in information retrieval, p.21-29, August 18-22, 1996, Zurich, Switzerland [doi>10.1145/243199.243206]
• 21: C. J. van Rijsbergen, The Geometry of Information Retrieval, Cambridge University Press, New York, NY, 2004

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Retrieval models

General Terms:
Theory

Keywords:
Poisson, binary retrieval model, language modelling