skip to main content
10.1145/1526709.1526711acmconferencesArticle/Chapter ViewAbstractPublication PagesthewebconfConference Proceedingsconference-collections
research-article

A dynamic bayesian network click model for web search ranking

Published: 20 April 2009 Publication History

Abstract

As with any application of machine learning, web search ranking requires labeled data. The labels usually come in the form of relevance assessments made by editors. Click logs can also provide an important source of implicit feedback and can be used as a cheap proxy for editorial labels. The main difficulty however comes from the so called position bias - urls appearing in lower positions are less likely to be clicked even if they are relevant. In this paper, we propose a Dynamic Bayesian Network which aims at providing us with unbiased estimation of the relevance from the click logs. Experiments show that the proposed click model outperforms other existing click models in predicting both click-through rate and relevance.

References

[1]
E. Agichtein, E. Brill, S. Dumais, and R. Ragno. Learning user interaction models for predicting web search result preferences. In Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval (SIGIR), pages 3--10, 2006.
[2]
M. Beal and Z. Ghahraman. Variational bayesian learning of directed graphical models with hidden variables. Bayesian Analysis, 1(4):793--832, 2006.
[3]
A. Broder. A taxonomy of web search. SIGIR Forum, 36(2):3--10, 2002.
[4]
C. Burges, T. Shaked, E. Renshaw, A. Lazier, M. Deeds, N. Hamilton, and G. Hullender. Learning to rank using gradient descent. In Proceedings of the 22nd international conference on Machine learning, pages 89--96, 2005.
[5]
Y. Cao, J. Xu, T.-Y. Liu, H. Li, Y. Huang, and H.-W. Hon. Adapting ranking svm to document retrieval. In Proceedings of the 29th annual international ACM SIGIR conference on Research and development in informat ion retrieval, 2006.
[6]
B. Carlin and T. Louis. Bayes and Empirical Bayes Methods for Data Analysis. Chapman & Hall/CRC, 2000.
[7]
B. Carterette and R. Jones. Evaluating search engines by modeling the relationship between relevance and clicks. In J. Platt, D. Koller, Y. Singer, and S. Roweis, editors, Advances in Neural Information Processing Systems 20, pages 217--224. MIT Press, 2008.
[8]
N. Craswell, O. Zoeter, M. Taylor, and B. Ramsey. An experimental comparison of click position-bias models. In WSDM '08: Proceedings of the international conference on Web search and web data mining, pages 87--94. ACM, 2008.
[9]
N. M. Dempster, A. Laird, and D. B. Rubin. Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society B, 39:185--197, 1977.
[10]
G. Dupret and B. Piwowarski. User browsing model to predict search engine click data from past observations. In SIGIR 08: Proceedings of the 31st Annual International Conference on Research and Development in Information Retrieval, 2008.
[11]
Z. Ghahramani. Learning dynamic bayesian network. In C. L. Giles and M. Gori, editors, Adaptive processing of temporal information, Lecture notes in artificial intelligence. Springer-Verlag, 1998.
[12]
K. Jarvelin and J. Kekalainen. Cumulated gain--based evaluation of IR techniques. ACM Transactions on Information Systems, 20(4):422--446, 2002.
[13]
T. Joachims. Optimizing search engines using clickthrough data. In ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD), pages 133--142, 2002.
[14]
T. Joachims. Evaluating retrieval peformance using clickthrough data. In Text mining, pages 79--96, 2003.
[15]
T. Joachims, L. A. Granka, B. Pan, H. Hembrooke, and G. Gay. Accurately interpreting clickthrough data as implicit feedback. In Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval, pages 154--161, 2005.
[16]
M. Richardson, E. Dominowska, and R. Ragno. Predicting clicks: estimating the click--through rate for new ads. In Proceedings of the 16th international conference on World Wide Web (WWW), pages 521--530, 2007.
[17]
M. Richardson, E. Dominowska, and R. Ragno. Predicting clicks: estimating the click-through rate for new ads. In WWW '07: Proceedings of the 16th international conference on World Wide Web, pages 521--530. ACM, 2007.
[18]
S. E. Robertson and S. Walker. Some simple effective approximations to the 2-poisson model for probabilistic weighted retrieval. In Proceedings of the 17th annual international ACM SIGIR conference on Research and development in informat ion retrieval, 1994.
[19]
V. Zhang and R. Jones. Comparing click logs and editorial labels for training query rewriting. In Query Log Analysis: Social And Technological Challenges. A workshop at the 16th International World Wide Web Conference, 2007.
[20]
Z. Zheng, H. Zha, T. Zhang, O. Chapelle, K. Chen, and G. Sun. A general boosting method and its application to learning ranking functions for web search. In Advances in Neural Information Processing Systems 20, pages 1697--1704. MIT Press, 2008.
[21]
D. Zhou, L. Bolelli, J. Li, C. L. Giles, and H. Zha. Learning user clicks in web search. In International Joint Conference on Artificial Intelligence (IJCAI07), 2007.

Cited By

View all
  • (2024)A bias study and an unbiased deep neural network for recommender systemsWeb Intelligence10.3233/WEB-23003622:1(15-29)Online publication date: 26-Mar-2024
  • (2024)A topic relevance-aware click model for web searchJournal of Intelligent & Fuzzy Systems10.3233/JIFS-23689446:4(8961-8974)Online publication date: 18-Apr-2024
  • (2024)Online and Offline Evaluation in Search ClarificationACM Transactions on Information Systems10.1145/368178643:1(1-30)Online publication date: 4-Nov-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
WWW '09: Proceedings of the 18th international conference on World wide web
April 2009
1280 pages
ISBN:9781605584874
DOI:10.1145/1526709

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 20 April 2009

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. click modeling
  2. click-through rate
  3. dynamic bayesian network
  4. ranking
  5. web search

Qualifiers

  • Research-article

Conference

WWW '09
Sponsor:

Acceptance Rates

Overall Acceptance Rate 1,899 of 8,196 submissions, 23%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)97
  • Downloads (Last 6 weeks)16
Reflects downloads up to 09 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)A bias study and an unbiased deep neural network for recommender systemsWeb Intelligence10.3233/WEB-23003622:1(15-29)Online publication date: 26-Mar-2024
  • (2024)A topic relevance-aware click model for web searchJournal of Intelligent & Fuzzy Systems10.3233/JIFS-23689446:4(8961-8974)Online publication date: 18-Apr-2024
  • (2024)Online and Offline Evaluation in Search ClarificationACM Transactions on Information Systems10.1145/368178643:1(1-30)Online publication date: 4-Nov-2024
  • (2024)Offline Evaluation of Set-Based Text-to-Image GenerationProceedings of the 2024 Annual International ACM SIGIR Conference on Research and Development in Information Retrieval in the Asia Pacific Region10.1145/3673791.3698424(42-53)Online publication date: 8-Dec-2024
  • (2024)Evaluation of Temporal Change in IR Test CollectionsProceedings of the 2024 ACM SIGIR International Conference on Theory of Information Retrieval10.1145/3664190.3672530(3-13)Online publication date: 2-Aug-2024
  • (2024)Towards Simulation-Based Evaluation of Recommender Systems with Carousel InterfacesACM Transactions on Recommender Systems10.1145/36437092:1(1-25)Online publication date: 30-Jan-2024
  • (2024)Learned Ranking Function: From Short-term Behavior Predictions to Long-term User SatisfactionProceedings of the 18th ACM Conference on Recommender Systems10.1145/3640457.3688184(1004-1009)Online publication date: 8-Oct-2024
  • (2024)Counteracting Duration Bias in Video Recommendation via Counterfactual Watch TimeProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3671817(4455-4466)Online publication date: 25-Aug-2024
  • (2024)USimAgent: Large Language Models for Simulating Search UsersProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657963(2687-2692)Online publication date: 10-Jul-2024
  • (2024)Neural Click Models for Recommender SystemsProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657939(2553-2558)Online publication date: 10-Jul-2024
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media