Dynamical information retrieval modelling: a portfolio-armed bandit machine approach

ABSTRACT

The dynamic nature of document relevance is largely ignored by traditional Information Retrieval (IR) models, which assume that scores (relevance) for documents given an information need are static. In this paper, we formulate a general Dynamical Information Retrieval problem, where we consider retrieval as a stochastic, controllable process. The ranking action continuously controls the retrieval system's dynamics and an optimal ranking policy is found that maximizes the overall users' satisfaction during each period. Through deriving the posterior probability of the documents evolving relevancy from user clicks, we can provide a plug-in framework for incorporating a number of click models, which can be combined with Multi-Armed Bandit theory and Portfolio Theory of IR to create a dynamic ranking rule that takes rank bias and click dependency into account. We verify the versatility of our algorithms in a number of experiments and demonstrate improved performance over strong baselines and as a result significant performance gains have been achieved.

References

1. Athans, M., and Falb, P. Optimal Control: An Introduction to the Theory and Its Applications. Dover Publications, 2006.
2. Auer, P., Cesa-Bianchi, N., and Fischer, P. Finite-time analysis of the multiarmed bandit problem. Mach. Learn. 47 (2002).
3. Craswell, N., Zoeter, O., Taylor, M., and Ramsey, B. An experimental comparison of click position-bias models. WSDM (2008), ACM.
4. Radlinski, F., Kleinberg, R., and Joachims, T. Learning diverse rankings with multi-armed bandits. ICML (2008), ACM.
5. Wang, J., and Zhu, J. Portfolio theory of information retrieval. SIGIR (2009), ACM.

Index Terms

• Information systems

  • Information retrieval

    • Document representation