skip to main content
10.1145/1571941.1571998acmconferencesArticle/Chapter ViewAbstractPublication PagesirConference Proceedingsconference-collections
research-article

Adaptation of offline vertical selection predictions in the presence of user feedback

Published: 19 July 2009 Publication History

Abstract

Web search results often integrate content from specialized corpora known as verticals. Given a query, one important aspect of aggregated search is the selection of relevant verticals from a set of candidate verticals. One drawback to previous approaches to vertical selection is that methods have not explicitly modeled user feedback. However, production search systems often record a variety of feedback information. In this paper, we present algorithms for vertical selection which adapt to user feedback. We evaluate algorithms using a novel simulator which models performance of a vertical selector situated in realistic query traffic.

References

[1]
]]S. Acharya, P. Krishnamurthy, K. Deshpande, T. Yan, and C.-C. Chang. A simulation framework for evaluating designs for sponsored search markets. In WWW 2007 Workshop on Sponsored Search Auctions, 2007.
[2]
]]J. Aitchison and S.M. Shen. Logistic-normal distributions: Some properties and uses. Biometrika, 67(2):261--272, August 1980.
[3]
]]J. Arguello, F. Diaz, J. Callan, and J.-F. Crespo. Sources of evidence for vertical selection. In SIGIR 2009, 2009.
[4]
]]L. Azzopardi, M. de Rijke, and K. Balog. Building simulated queries for known-item topics: an analysis using six european languages. In SIGIR 2007, pages 455--462, 2007.
[5]
]]S.M. Beitzel, E.C. Jensen, O. Frieder, D.D. Lewis, A. Chowdhury, and A. Kolcz. Improving automatic query classification via semi-supervised learning. In ICDM 2005, pages 42--49, 2005.
[6]
]]J. Callan. Distributed information retrieval. In W.B. Croft, editor, Advances in Information Retrieval. 2000.
[7]
]]M. Ciaramita, V. Murdock, and V. Plachouras. Online learning from click data for sponsored search. In WWW 2008, pages 227--236, 2008.
[8]
]]M.D. Cooper. A simulation model of an information retrieval system. Information Storage and Retrieval, 9(1):13--32, 1973.
[9]
]]S. Cronen-Townsend, Y. Zhou, and W.B. Croft. Predicting query performance. In SIGIR 2002, pages 299--306, 2002.
[10]
]]F. Diaz. Integration of news content into web results. In WSDM 2009, 2009.
[11]
]]J.-M. Griffiths. The computer simulation of information retrieval systems. PhD thesis, University College London, 1977.
[12]
]]T. Joachims. Optimizing search engines using clickthrough data. In KDD 2002, pages 133--142, 2002.
[13]
]]R. Jones and K.L. Klinkner. Beyond the session timeout: automatic hierarchical segmentation of search topics in query logs. In CIKM 2008, pages 699--708, 2008.
[14]
]]P.J. Lenk and B.D. Floyd. Dynamically updating relevance judgements in probabilistic information systems via users' feedback. Management Science, 34(12):1450--1459, December 1988.
[15]
]]X. Li, Y.-Y. Wang, and A. Acero. Learning query intent from regularized click graphs. In SIGIR 2008, pages 339--346, 2008.
[16]
]]D. Metzler, S.T. Dumais, and C. Meek. Similarity measures for short segments of text. In ECIR 2007, pages 16--27, 2007.
[17]
]]J. Mostafa, S. Mukhopadhyay, and M. Palakal. Simulation studies of different dimensions of users' interests and their impact on user modeling and information filtering. Information Retrieval, 6:199--223, April 2003.
[18]
]]V. Murdock and M. Lalmas, editors. Proceedings of the SIGIR Workshop on Aggregated Search, 2008.
[19]
]]F. Radlinski, R. Kleinberg, and T. Joachims. Learning diverse rankings with multi-armed bandits. In ICML 2008, pages 784--791, 2008.
[20]
]]F. Radlinski, M. Kurup, and T. Joachims. How does clickthrough data reflect retrieval quality? In CIKM 2008, pages 43--52, 2008.
[21]
]]M. Richardson, E. Dominowska, and R. Ragno. Predicting clicks: estimating the click-through rate for new ads. In WWW 2007, pages 521--530, 2007.
[22]
]]M. Sahami and T.D. Heilman. A web-based kernel function for measuring the similarity of short text snippets. In WWW 2006, pages 377--386, 2006.
[23]
]]D. Shen, J.-T. Sun, Q. Yang, and Z. Chen. Building bridges for web query classification. In SIGIR 2006, pages 131--138, 2006.
[24]
]]R. Sutton and A. Barto. Reinforcement Learning. 1998.
[25]
]]J. Tague, M. Nelson, and H. Wu. Problems in the simulation of bibliographic retrieval systems. In SIGIR 1980, pages 236--255, 1980.
[26]
]]R.W. White, I. Ruthven, J.M. Jose, and C.J. van Rijsbergen. Evaluating implicit feedback models using searcher simulations. TOIS, 23(3):325--361, July 2005.

Cited By

View all
  • (2024)Analysing Utterances in LLM-Based User Simulation for Conversational SearchACM Transactions on Intelligent Systems and Technology10.1145/365004115:3(1-22)Online publication date: 5-Mar-2024
  • (2022)A Novel Probabilistic Graphical Model-Based Click Model for Vertical SearchJournal of the Korean Institute of Industrial Engineers10.7232/JKIIE.2022.48.2.13848:2(138-150)Online publication date: 15-Apr-2022
  • (2019)An Analysis Study of Vertical Selection Task in Aggregated SearchProcedia Computer Science10.1016/j.procs.2019.01.021148(171-180)Online publication date: 2019
  • Show More Cited By

Index Terms

  1. Adaptation of offline vertical selection predictions in the presence of user feedback

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    SIGIR '09: Proceedings of the 32nd international ACM SIGIR conference on Research and development in information retrieval
    July 2009
    896 pages
    ISBN:9781605584836
    DOI:10.1145/1571941
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 19 July 2009

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. aggregated search
    2. distributed information retrieval
    3. resource selection
    4. simulation
    5. user feedback
    6. vertical selection

    Qualifiers

    • Research-article

    Conference

    SIGIR '09
    Sponsor:

    Acceptance Rates

    Overall Acceptance Rate 792 of 3,983 submissions, 20%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)4
    • Downloads (Last 6 weeks)1
    Reflects downloads up to 18 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)Analysing Utterances in LLM-Based User Simulation for Conversational SearchACM Transactions on Intelligent Systems and Technology10.1145/365004115:3(1-22)Online publication date: 5-Mar-2024
    • (2022)A Novel Probabilistic Graphical Model-Based Click Model for Vertical SearchJournal of the Korean Institute of Industrial Engineers10.7232/JKIIE.2022.48.2.13848:2(138-150)Online publication date: 15-Apr-2022
    • (2019)An Analysis Study of Vertical Selection Task in Aggregated SearchProcedia Computer Science10.1016/j.procs.2019.01.021148(171-180)Online publication date: 2019
    • (2017)Aggregated SearchFoundations and Trends in Information Retrieval10.1561/150000005210:5(365-502)Online publication date: 6-Mar-2017
    • (2017)Re-Finding Behaviour in Vertical DomainsACM Transactions on Information Systems10.1145/297559035:3(1-30)Online publication date: 5-Jun-2017
    • (2016)The Effects of Aggregated Search Coherence on Search BehaviorACM Transactions on Information Systems10.1145/293574735:1(1-30)Online publication date: 22-Sep-2016
    • (2016)Online training of concept detectors for image retrieval using streaming clickthrough dataEngineering Applications of Artificial Intelligence10.1016/j.engappai.2016.01.01751:C(150-162)Online publication date: 1-May-2016
    • (2015)Gathering Additional Feedback on Search Results by Multi-Armed Bandits with Respect to Production RankingProceedings of the 24th International Conference on World Wide Web10.1145/2736277.2741104(1177-1187)Online publication date: 18-May-2015
    • (2015)Online Experimentation for Information RetrievalInformation Retrieval10.1007/978-3-319-25485-2_2(21-41)Online publication date: 10-Dec-2015
    • (2015)Improving Aggregated Search CoherenceAdvances in Information Retrieval10.1007/978-3-319-16354-3_3(25-36)Online publication date: 2015
    • Show More Cited By

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media