Joint relevance and freshness learning from clickthroughs for news search

ABSTRACT

In contrast to traditional Web search, where topical relevance is often the main selection criterion, news search is characterized by the increased importance of freshness. However, the estimation of relevance and freshness, and especially the relative importance of these two aspects, are highly specific to the query and the time when the query was issued. In this work, we propose a unified framework for modeling the topical relevance and freshness, as well as their relative importance, based on click logs. We use click statistics and content analysis techniques to define a set of temporal features, which predict the right mix of freshness and relevance for a given query. Experimental results on both historical click data and editorial judgments demonstrate the effectiveness of the proposed approach.

References

1. D. Agarwal, B.-C. Chen, P. Elango, and X. Wang. Click shaping to optimize multiple objectives. In KDD, 2011.
2. E. Agichtein, E. Brill, S. Dumais, and R. Ragno. Learning user interaction models for predicting web search result preferences. In SIGIR, 2006.
3. R. Baeza-Yates, B. Ribeiro-Neto, et al. Modern information retrieval, volume 463. ACM press New York, 1999.
4. S. Brin and L. Page. The anatomy of a large-scale hypertextual web search engine. Computer networks and ISDN systems, 30(1--7):107--117, 1998.
5. Z. Cao, T. Qin, T. Liu, M. Tsai, and H. Li. Learning to rank: from pairwise approach to listwise approach. In ICML, pages 129--136. ACM, 2007.
6. H. Chernoff and E. Lehmann. The use of maximum likelihood estimates in $\chi$2 tests for goodness of fit. The Annals of Mathematical Statistics, pages 579--586, 1954.
7. N. Dai, M. Shokouhi, and B. D. Davison. Learning to rank for freshness and relevance. In SIGIR, pages 95--104, 2011.
8. A. Dong, Y. Chang, Z. Zheng, G. Mishne, J. Bai, R. Zhang, K. Buchner, C. Liao, and F. Diaz. Towards recency ranking in web search. In WSDM, pages 11--20, 2010.
9. A. Dong, R. Zhang, P. Kolari, J. Bai, F. Diaz, Y. Chang, Z. Zheng, and H. Zha. Time is of the essence: improving recency ranking using twitter data. In WWW, 2010.
10. M. Efron and G. Golovchinsky. Estimation methods for ranking recent information. In SIGIR, pages 495--504, 2011.
11. H. Fang, T. Tao, and C. Zhai. A formal study of information retrieval heuristics. In SIGIR, 2004.
12. K. J\"arvelin and J. Kek\"al\"ainen. Cumulated gain-based evaluation of ir techniques. ACM TOIS, 20(4):422--446, 2002.
13. T. Joachims. Optimizing search engines using clickthrough data. In KDD, pages 133--142, 2002.
14. T. Joachims, L. Granka, B. Pan, H. Hembrooke, and G. Gay. Accurately interpreting clickthrough data as implicit feedback. In SIGIR, pages 154--161, 2005.
15. K. Jones, S. Walker, and S. Robertson. A probabilistic model of information retrieval: development and comparative experiments. Information Processing and Management, 36(6):779--808, 2000.
16. N. Kanhabua and K. Nørvåg. Determining time of queries for re-ranking search results. Research and Advanced Technology for Digital Libraries, pages 261--272, 2010.
17. A. Kulkarni, J. Teevan, K. Svore, and S. Dumais. Understanding temporal query dynamics. In WSDM, 2011.
18. L. Li, W. Chu, J. Langford, and X. Wang. Unbiased offline evaluation of contextual-bandit-based news article recommendation algorithms. In Proceedings of the fourth ACM WSDM '11, pages 297--306, 2011.
19. X. Li and W. Croft. Time-based language models. In CIKM, pages 469--475, 2003.
20. T. Liu. Learning to rank for information retrieval. Foundations and Trends in Information Retrieval, 3(3):225--331, 2009.
21. T. Moon, L. Li, W. Chu, C. Liao, Z. Zheng, and Y. Chang. Online learning for recency search ranking using real-time user feedback. In CIKM, pages 1501--1504, 2010.
22. G. Salton and M. McGill. Introduction to modern information retrieval. McGraw-Hill, Inc., 1986.
23. K. M. Svore, M. N. Volkovs, and C. J. Burges. Learning to rank with multiple objective functions. In WWW, 2011.
24. C. Zhai and J. Lafferty. A study of smoothing methods for language models applied to ad hoc information retrieval. In SIGIR, pages 334--342, 2001.
25. Z. Zheng, K. Chen, G. Sun, and H. Zha. A regression framework for learning ranking functions using relative relevance judgments. In SIGIR, pages 287--294, 2007.

Index Terms

• Information systems

  • Information retrieval

    • Retrieval models and ranking