skip to main content
10.1145/3176349.3176400acmconferencesArticle/Chapter ViewAbstractPublication PagesirConference Proceedingsconference-collections
research-article

A Study of Immediate Requery Behavior in Search

Published:01 March 2018Publication History

ABSTRACT

When search results fail to satisfy users» information needs, users often reformulate their search query in the hopes of receiving better results. In many cases, users immediately requery without clicking on any search results. In this paper, we report on a user study designed to investigate the rate at which users immediately reformulate at different levels of search quality. We had users search for answers to questions as we manipulated the placement of the only relevant document in a ranked list of search results. We show that as the quality of search results decreases, the probability of immediately requerying increases. We find that users can quickly decide to immediately reformulate, and the time to immediately reformulate appears to be independent of the quality of the search results. Finally, we show that there appears to be two types of users. One group has a high probability of immediately reformulating and the other is unlikely to immediately reformulate unless no relevant documents can be found in the search results. While requerying takes time, it is the group of users who are more likely to immediately requery that are able to able find answers to questions the fastest.

References

  1. Anne Aula, Päivi Majaranta, and Kari-Jouko Räihä. 2005. Eye-Tracking Reveals the Personal Styles for Search Result Evaluation. In Human-Computer Interaction -- INTERACT 2005 ( LNCS), Vol. 3585, 1058--1061. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Leif Azzopardi. 2011. The economics in interactive information retrieval. In SIGIR, 15--24. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Leif Azzopardi, Kalervo Järvelin, Jaap Kamps, and Mark D. Smucker. 2011. Report on the SIGIR 2010 workshop on the simulation of interaction. SIGIR Forum 44 (January 2011), 35--47. Issue 2. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Feza Baskaya, Heikki Keskustalo, and Kalervo Järvelin. 2012. Time Drives Interaction: Simulating Sessions in Diverse Searching Environments. In SIGIR, 105--114. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Ben Carterette, Evangelos Kanoulas, and Emine Yilmaz. 2011. Simulating simple user behavior for system effectiveness evaluation. In CIKM, 611--620. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Charles L.A. Clarke, Luanne Freund, Mark D. Smucker, and Emine Yilmaz. 2013. Report on the SIGIR 2013 Workshop on Modeling User Behavior for Information Retrieval Evaluation (MUBE 2013). SIGIR Forum 47, 2 (Jan. 2013), 84--95. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Edward Cutrell and Zhiwei Guan. 2007. What are you looking for?: an eyetracking study of information usage in web search. In SIGCHI, 407--416. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Arjen P. de Vries, Gabriella Kazai, and Mounia Lalmas. 2004. Tolerance to Irrelevance: A User-effort Oriented Evaluation of Retrieval Systems without Predefined Retrieval Unit. In RIAO, 463--473. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Abdigani Diriye, Ryen White, Georg Buscher, and Susan Dumais. 2012. Leaving so soon?: understanding and predicting web search abandonment rationales. In CIKM, 1025--1034. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Susan T. Dumais, Georg Buscher, and Edward Cutrell. 2010. Individual differences in gaze patterns for web search. In IIiX, 185--194. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Laura A. Granka, Thorsten Joachims, and Geri Gay. 2004. Eye-tracking analysis of user behavior in WWW search. In SIGIR, 478--479. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Marti A. Hearst. 2009. Search User Interfaces (1st ed.). Cambridge University Press. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Jeff Huang, Ryen W White, and Susan Dumais. 2011. No clicks, no problem: using cursor movements to understand and improve search. In SIGCHI, 1225--1234. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Thorsten Joachims, Laura Granka, Bing Pan, Helene Hembrooke, and Geri Gay. 2005. Accurately interpreting clickthrough data as implicit feedback. In SIGIR, 154--161. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Thorsten Joachims, Laura Granka, Bing Pan, Helene Hembrooke, Filip Radlinski, and Geri Gay. 2007. Evaluating the Accuracy of Implicit Feedback from Clicks and Query Reformulations in Web Search. ACM TOIS 25, 2 (2007), 1--27. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Thorsten Joachims and Filip Radlinski. 2007. Search Engines That Learn from Implicit Feedback. Computer 40, 8 (Aug. 2007), 34--40. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Kerstin Klöckner, Nadine Wirschum, and Anthony Jameson. 2004. Depth- and breadth-first processing of search result lists. In SIGCHI extended abstracts, 1539-- 1539. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Jane Li, Scott Huffman, and Akihito Tokuda. 2009. Good Abandonment in Mobile and PC Internet Search. In SIGIR, 43--50. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Lori Lorigo, Maya Haridasan, Hrönn Brynjarsdóttir, Ling Xia, Thorsten Joachims, Geri Gay, Laura Granka, Fabio Pellacini, and Bing Pan. 2008. Eye tracking and online search: Lessons learned and challenges ahead. JASIS 59, 7 (2008), 1041-- 1052. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Lori Lorigo, Maya Haridasan, Hrönn Brynjarsdóttir, Ling Xia, Thorsten Joachims, Geri Gay, Laura Granka, Fabio Pellacini, and Bing Pan. 2008. Eye tracking and online search: Lessons learned and challenges ahead. JAIST 59, 7 (2008), 1041-- 1052. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Andrew MacFarlane, George Buchanan, Areej Al-Wabil, Gennady Andrienko, and Natalia Andrienko. 2017. Visual Analysis of Dyslexia on Search. In CHIIR, 285--288. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. David Maxwell, Leif Azzopardi, Kalervo Järvelin, and Heikki Keskustalo. 2015. Searching and Stopping: An Analysis of Stopping Rules and Strategies. In CIKM, 313--322. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Teemu Pääkkönen, Jaana Kekäläinen, Heikki Keskustalo, Leif Azzopardi, David Maxwell, and Kalervo Järvelin. 2017. Validating simulated interaction for retrieval evaluation. Information Retrieval (2017), 1--25. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Filip Radlinski, Madhu Kurup, and Thorsten Joachims. 2008. How Does Clickthrough Data Reflect Retrieval Quality?. In CIKM, 43--52. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Mark D. Smucker and Charles L.A. Clarke. 2012. Time-based calibration of effectiveness measures. In SIGIR, 95--104. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Mark D. Smucker and Charles L.A. Clarke. 2016. Modeling Optimal Switching Behavior. In CHIIR, 317--320. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Mark D. Smucker and Charles L. A. Clarke. 2012. Modeling User Variance in Time-Biased Gain. In HCIR. ACM, 1--10. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Sofia Stamou and Efthimis N Efthimiadis. 2009. Queries without clicks: Successful or failed searches. In SIGIR 2009 Workshop on the Future of IR Evaluation. 13--14.Google ScholarGoogle Scholar
  29. Sofia Stamou and Efthimis N. Efthimiadis. 2010. Interpreting User Inactivity on Search Results. In ECIR, 100--113. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Paul Thomas, Alistair Moffat, Peter Bailey, and Falk Scholer. 2014. Modeling Decision Points in User Search Behavior. In IIiX, 239--242. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Wan-Ching Wu and Diane Kelly. 2014. Online search stopping behaviors: An investigation of query abandonment and task stopping. Proceedings of the Association for Information Science and Technology 51, 1 (2014), 1--10.Google ScholarGoogle ScholarCross RefCross Ref
  32. Wan-Ching Wu, Diane Kelly, and Avneesh Sud. 2014. Using information scent and need for cognition to understand online search behavior. In SIGIR, 557--566. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. A Study of Immediate Requery Behavior in Search

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in
    • Published in

      cover image ACM Conferences
      CHIIR '18: Proceedings of the 2018 Conference on Human Information Interaction & Retrieval
      March 2018
      402 pages
      ISBN:9781450349253
      DOI:10.1145/3176349

      Copyright © 2018 ACM

      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 1 March 2018

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • research-article

      Acceptance Rates

      CHIIR '18 Paper Acceptance Rate22of57submissions,39%Overall Acceptance Rate55of163submissions,34%

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader