skip to main content
10.1145/1835449.1835606acmconferencesArticle/Chapter ViewAbstractPublication PagesirConference Proceedingsconference-collections
poster

Evaluating whole-page relevance

Published: 19 July 2010 Publication History

Abstract

Whole page relevance defines how well the surface-level repre-sentation of all elements on a search result page and the corre-sponding holistic attributes of the presentation respond to users' information needs. We introduce a method for evaluating the whole-page relevance of Web search engine results pages. Our key contribution is that the method allows us to investigate aspects of component relevance that are difficult or impossible to judge in isolation. Such aspects include component-level information redundancy and cross-component coherence. The method we describe complements traditional document relevance measurement, affords comparative relevance assessment across multiple search engines, and facilitates the study of important factors such as brand presentation effects and component-level quality.

References

[1]
Clarke, C.L.A., Agichtein, E., Dumais, S., and White, R.W. (2007). The influence of caption features on clickthrough patterns in web search. Proc. SIGIR, 135--142.
[2]
Cleverdon, C.W. (1960). ASLIB Cranfield research project on the comparative efficiency of indexing systems. ASLIB Proceedings, XII, 421--431.
[3]
Kohavi, R., Henne, R., and Sommerfield, D. (2007). Practical guide to controlled experiments on the web: listen to your customers not to the HiPPO. Proc. SIGKDD, 959--967.
[4]
Turpin, A., Scholer, F., Järvelin, K., Wu, M., and Culpepper, J.S. (2009). Including summaries in system evaluation. Proc. SIGIR, 508--515.
[5]
Voorhees, E.M. (2008). On test collections for adaptive information retrieval. Information Processing and Management, 44(6): 1879--1885.
[6]
Voorhees, E.M. and Harman, D. (2005). TREC: Experiment and Evaluation in Information Retrieval. MIT Press.

Cited By

View all
  • (2022)Batch Evaluation Metrics in Information Retrieval: Measures, Scales, and MeaningIEEE Access10.1109/ACCESS.2022.321166810(105564-105577)Online publication date: 2022
  • (2021)Mixed Method Development of Evaluation MetricsProceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining10.1145/3447548.3470802(4070-4071)Online publication date: 14-Aug-2021
  • (2020)Data-Driven Evaluation Metrics for Heterogeneous Search Engine Result PagesProceedings of the 2020 Conference on Human Information Interaction and Retrieval10.1145/3343413.3377959(213-222)Online publication date: 14-Mar-2020
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
SIGIR '10: Proceedings of the 33rd international ACM SIGIR conference on Research and development in information retrieval
July 2010
944 pages
ISBN:9781450301534
DOI:10.1145/1835449
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 19 July 2010

Check for updates

Author Tags

  1. evaluation
  2. measurement
  3. web search relevance

Qualifiers

  • Poster

Conference

SIGIR '10
Sponsor:

Acceptance Rates

SIGIR '10 Paper Acceptance Rate 87 of 520 submissions, 17%;
Overall Acceptance Rate 792 of 3,983 submissions, 20%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)7
  • Downloads (Last 6 weeks)1
Reflects downloads up to 18 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2022)Batch Evaluation Metrics in Information Retrieval: Measures, Scales, and MeaningIEEE Access10.1109/ACCESS.2022.321166810(105564-105577)Online publication date: 2022
  • (2021)Mixed Method Development of Evaluation MetricsProceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining10.1145/3447548.3470802(4070-4071)Online publication date: 14-Aug-2021
  • (2020)Data-Driven Evaluation Metrics for Heterogeneous Search Engine Result PagesProceedings of the 2020 Conference on Human Information Interaction and Retrieval10.1145/3343413.3377959(213-222)Online publication date: 14-Mar-2020
  • (2020)Search Engine Similarity Analysis: A Combined Content and Rankings ApproachWeb Information Systems Engineering – WISE 202010.1007/978-3-030-62008-0_2(21-37)Online publication date: 20-Oct-2020
  • (2018)Ranking for Relevance and Display Preferences in Complex Presentation LayoutsThe 41st International ACM SIGIR Conference on Research & Development in Information Retrieval10.1145/3209978.3209992(845-854)Online publication date: 27-Jun-2018
  • (2018)Optimizing Whole-Page Presentation for Web SearchACM Transactions on the Web10.1145/320446112:3(1-25)Online publication date: 17-Jul-2018
  • (2018)Utilizing Entities for an Enhanced Search ExperienceEntity-Oriented Search10.1007/978-3-319-93935-3_9(299-336)Online publication date: 3-Oct-2018
  • (2017)Aggregated SearchFoundations and Trends in Information Retrieval10.1561/150000005210:5(365-502)Online publication date: 6-Mar-2017
  • (2017)Incorporating User Expectations and Behavior into the Measurement of Search EffectivenessACM Transactions on Information Systems10.1145/305276835:3(1-38)Online publication date: 5-Jun-2017
  • (2016)Nonlinear Composite Search ResultsProceedings of the 2016 ACM on Conference on Human Information Interaction and Retrieval10.1145/2854946.2854956(345-347)Online publication date: 13-Mar-2016
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media