skip to main content
10.1145/1835449.1835458acmconferencesArticle/Chapter ViewAbstractPublication PagesirConference Proceedingsconference-collections
research-article

Predicting searcher frustration

Published: 19 July 2010 Publication History

Abstract

When search engine users have trouble finding information, they may become frustrated, possibly resulting in a bad experience (even if they are ultimately successful). In a user study in which participants were given difficult information seeking tasks, half of all queries submitted resulted in some degree of self-reported frustration. A third of all successful tasks involved at least one instance of frustration. By modeling searcher frustration, search engines can predict the current state of user frustration and decide when to intervene with alternative search strategies to prevent the user from becoming more frustrated, giving up, or switching to another search engine. We present several models to predict frustration using features extracted from query logs and physical sensors. We are able to predict frustration with a mean average precision of 65% from the physical sensors, and 87% from the query log features.

References

[1]
A. Al-Maskari, M. Sanderson, and P. Clough. The relationship between IR effectiveness measures and user satisfaction.In Proc. of SIGIR, pages 773--774, 2007.
[2]
D. Bilal and J. Kirby. Differences and similarities in information seeking: children and adults as Web users. Info. Proc. and Mngt., 38(5):649--670, 2002.
[3]
I. Ceaparu, J. Lazar, K. Bessiere, J. Robinson, and B. Shneiderman. Determining Causes and Severity of End-User Frustration. Intl. J. of HCI, 17(3):333--356, 2004.
[4]
D. G. Cooper, I. Arroyo, B. P. Woolf, K. Muldner, W. Burleson, and R. Christopherson. Sensor model student self concept in the classroom. In Proc. of UMAP, pages 30--41, June 2009.
[5]
S. D'Mello, R. Picard, and A. Graesser. Toward an affect-sensitive AutoTutor. IEEE Intel. Sys., pages 53--61, 2007.
[6]
D. Downey, S. Dumais, and E. Horvitz. Models of searching and browsing: languages, studies, and applications. In Proc. of ICAI, pages 2740--2747, 2007.
[7]
A. Druin, E. Foss, L. Hatley, E. Golub, M. L. Guha, J. Fails, and H. Hutchinson. How children search the internet with keyword interfaces. In Proc. of IDC, pages 89--96, 2009.
[8]
S. Fox, K. Karnawat, M. Mydland, S. Dumais, and T.White. Evaluating implicit measures to improve web search. ACM Trans. on Info. Sys., 23(2):147--168, 2005.
[9]
A. Hassan, R. Jones, and K. L. Klinkner. Beyond dcg: User behavior as a predictor of a successful search. In Proc. of WSDM, pages 221--230, 2010.
[10]
S. Huffman and M. Hochster. How well does result relevance predict session satisfaction? In Proc. of SIGIR, pages 567--574, 2007.
[11]
A. Kapoor, W. Burleson, and R. Picard. Automatic prediction of frustration. Intl. J. of Human-Computer Studies, 65 (8):724--736, 2007.
[12]
C. C. Kuhlthau. Inside the search process: Information seeking from the user's perspective. JASIST, 42(5):361--371, 1991.
[13]
D. J. Lawrie. Language models for hierarchical summarization. PhD thesis, 2003.
[14]
M. Smucker, J. Allan, and B. Carterette. A comparison of statistical significance tests for information retrieval evaluation. In Proc. of CIKM, pages 623--632, 2007.
[15]
R. White, J. Jose, and I. Ruthven. An implicit feedback approach for interactive information retrieval. Info. Proc. and Mngt., 42(1):166--190, 2006.
[16]
R. W. White and S. T. Dumais. Characterizing and predicting search engine switching behavior. In Proc. of CIKM, pages 87--96, 2009.
[17]
I. Xie and C. Cool. Understanding help seeking within the context of searching digital libraries. JASIST, 60(3):477--494, 2009.

Cited By

View all

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
SIGIR '10: Proceedings of the 33rd international ACM SIGIR conference on Research and development in information retrieval
July 2010
944 pages
ISBN:9781450301534
DOI:10.1145/1835449
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 19 July 2010

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. query logs
  2. searcher frustration
  3. user modeling

Qualifiers

  • Research-article

Conference

SIGIR '10
Sponsor:

Acceptance Rates

SIGIR '10 Paper Acceptance Rate 87 of 520 submissions, 17%;
Overall Acceptance Rate 792 of 3,983 submissions, 20%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)48
  • Downloads (Last 6 weeks)4
Reflects downloads up to 18 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Improving Educators’ Search Engine Experience: A Quantitative Analysis of Search TermsIEEE Access10.1109/ACCESS.2024.339342312(69076-69086)Online publication date: 2024
  • (2024)Determining the causes of user frustration in the case of conversational chatbotsBehaviour & Information Technology10.1080/0144929X.2024.2362956(1-19)Online publication date: 13-Jun-2024
  • (2024)Improving searcher struggle detection via the reversal theoryDiscover Computing10.1007/s10791-024-09492-z27:1Online publication date: 19-Dec-2024
  • (2023)Taking Search to TaskProceedings of the 2023 Conference on Human Information Interaction and Retrieval10.1145/3576840.3578288(1-13)Online publication date: 19-Mar-2023
  • (2023)Exploring the Use of the Delphi Method in Visual Search Service Effectiveness Measurement Research in e-Commerce Platform2023 International Conference on Software and System Engineering (ICoSSE)10.1109/ICoSSE58936.2023.00017(52-56)Online publication date: Apr-2023
  • (2023)The Effects of Patient-Centered Communication on Emotional Health: Examining the Roles of Self-Efficacy, Information Seeking Frustration, and Social Media UseJournal of Health Communication10.1080/10810730.2023.220853728:6(349-359)Online publication date: 5-May-2023
  • (2023)Development of a Visual Search Service Effectiveness Scale for Assessing Image Search Effectiveness: A Behavioral and Technological PerspectiveInternational Journal of Human–Computer Interaction10.1080/10447318.2023.219753540:14(3717-3731)Online publication date: 17-Apr-2023
  • (2022)AR-AI Tools as a Response to High Employee Turnover and Shortages in Manufacturing during Regular, Pandemic, and War TimesSustainability10.3390/su1411672914:11(6729)Online publication date: 31-May-2022
  • (2022)Intent Disambiguation for Task-oriented Dialogue SystemsProceedings of the 31st ACM International Conference on Information & Knowledge Management10.1145/3511808.3557516(5079-5080)Online publication date: 17-Oct-2022
  • (2022)Dynamic Information Retrieval ModelingundefinedOnline publication date: 10-Mar-2022
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media