Article

Context-based question-answering evaluation

Authors:

SIGIR '04: Proceedings of the 27th annual international ACM SIGIR conference on Research and development in information retrieval

Pages 508 - 509

https://doi.org/10.1145/1008992.1009094

Published: 25 July 2004 Publication History

Get Access

Abstract

In this poster, we will present the results of efforts we have undertaken to conduct evaluations of a QA system in a real world environment and to understand the nature of the dimensions on which users evaluate QA systems when given full reign to comment on whatever dimensions they deem important.

References

[1]

Voorhees, E.M. Overview of the TREC-8 Question Answering Track Report. In Proceedings of TREC-8, 77-82. Gaithersburg, MD, NIST, 2000.

Google Scholar

[2]

Voorhees, E. M. & Tice, D. M. Implementing a Question Answering Evaluation. In Proceedings of LREC'2000 Workshop on Using Evaluation within HLT Programs: Results and Trends. 2000.

Google Scholar

[3]

Diekema, A.R., Yilmazel, O., Chen, J., Harwell, S., He, L., and Liddy E.D. Finding Answers to Complex Questions. In Maybury, M. (Ed.) New Directions in Question Answering. AAAI-MIT Press. In Press.

Google Scholar

[4]

Diekema, A., Chen, J., Harwell, S., Yilmazel, O., He, L., and Liddy, E.D. What Do You Mean? Finding Answers to Complex Questions. In Proceedings of New Directions in Question Answering. AAAI Spring Symposium, March 24--26, 2003.

Google Scholar

Cited By

View all

Shouyu LWei JYuanwei YKun ZYubin HYingchen L(2023)Question Answering Retrieval Method for Knowledge-Based with Dynamic Programming Algorithm2023 2nd International Conference for Innovation in Technology (INOCON)10.1109/INOCON57975.2023.10101356(1-5)Online publication date: 3-Mar-2023
https://doi.org/10.1109/INOCON57975.2023.10101356
Kelly DKantor PMorse EScholtz JSun Y(2009)Questionnaires for eliciting evaluation data from users of interactive question answering systemsNatural Language Engineering10.1017/S135132490800493215:1(119-141)Online publication date: 1-Jan-2009
https://dl.acm.org/doi/10.1017/S1351324908004932
Sun MChai J(2007)Discourse processing for context question answering based on linguistic knowledgeKnowledge-Based Systems10.1016/j.knosys.2007.04.00520:6(511-526)Online publication date: 1-Aug-2007
https://dl.acm.org/doi/10.1016/j.knosys.2007.04.005
Show More Cited By

Index Terms

Context-based question-answering evaluation
1. Information systems
  1. Information retrieval
    1. Evaluation of retrieval results

Recommendations

Quality-aware collaborative question answering: methods and evaluation
WSDM '09: Proceedings of the Second ACM International Conference on Web Search and Data Mining

Community Question Answering (QA) portals contain questions and answers contributed by hundreds of millions of users. These databases of questions and answers are of great value if they can be used directly to answer questions from any user. In this ...
Question Answering System Based on Web
ICICTA '12: Proceedings of the 2012 Fifth International Conference on Intelligent Computation Technology and Automation

This paper summarizes the classification, implementation and evaluation of question answering system (QA). QA is divided into four categories: chat robot, QA based knowledge base, QA retrieval system and QA based on free text. Web QA system is composed ...
Interactive natural language question answering over knowledge graphs
Abstract
As many real-world data are constructed into knowledge graphs, providing effective and convenient query techniques for end users is an urgent and important task. Although structured query languages, such as SPARQL, offer a powerful ...

Comments

Information & Contributors

Information

Published In

SIGIR '04: Proceedings of the 27th annual international ACM SIGIR conference on Research and development in information retrieval

July 2004

624 pages

ISBN:1581138814

DOI:10.1145/1008992

General Chair:
Mark Sanderson
University of Sheffield (UK)
,
Program Chairs:
Kalervo Järvelin
University of Tampere (Finland)
,
James Allan
University of Massachusetts (USA)
,
Peter Bruza
Distributed Systems Technology Centre (Australia)

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 July 2004

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

SIGIR04

Sponsor:

SIGIR04: The 27th ACM/SIGIR International Symposium on Information Retrieval 2004

July 25 - 29, 2004

Sheffield, United Kingdom

Acceptance Rates

Overall Acceptance Rate 792 of 3,983 submissions, 20%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

8
Total Citations
View Citations
563
Total Downloads

Downloads (Last 12 months)3
Downloads (Last 6 weeks)2

Reflects downloads up to 18 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Shouyu LWei JYuanwei YKun ZYubin HYingchen L(2023)Question Answering Retrieval Method for Knowledge-Based with Dynamic Programming Algorithm2023 2nd International Conference for Innovation in Technology (INOCON)10.1109/INOCON57975.2023.10101356(1-5)Online publication date: 3-Mar-2023
https://doi.org/10.1109/INOCON57975.2023.10101356
Kelly DKantor PMorse EScholtz JSun Y(2009)Questionnaires for eliciting evaluation data from users of interactive question answering systemsNatural Language Engineering10.1017/S135132490800493215:1(119-141)Online publication date: 1-Jan-2009
https://dl.acm.org/doi/10.1017/S1351324908004932
Sun MChai J(2007)Discourse processing for context question answering based on linguistic knowledgeKnowledge-Based Systems10.1016/j.knosys.2007.04.00520:6(511-526)Online publication date: 1-Aug-2007
https://dl.acm.org/doi/10.1016/j.knosys.2007.04.005
Kelly DWacholder NRittman RSun YKantor PSmall SStrzalkowski T(2007)Using interview data to identify evaluation criteria for interactive, analytical question‐answering systemsJournal of the American Society for Information Science and Technology10.1002/asi.2057558:7(1032-1043)Online publication date: 21-Mar-2007
https://doi.org/10.1002/asi.20575
Kelly DKantor PMorse EScholtz JSun YWebb N(2006)User-centered evaluation of interactive question answering systemsProceedings of the Interactive Question Answering Workshop at HLT-NAACL 200610.5555/1641579.1641586(49-56)Online publication date: 8-Jun-2006
https://dl.acm.org/doi/10.5555/1641579.1641586
McCracken NDiekema AIngersoll GHarwell SAllen EYilmazel OLiddy EWebb N(2006)Modeling reference interviews as a basis for improving automatic QA systemsProceedings of the Interactive Question Answering Workshop at HLT-NAACL 200610.5555/1641579.1641582(17-24)Online publication date: 8-Jun-2006
https://dl.acm.org/doi/10.5555/1641579.1641582
Sun MChai JEdmonds ERiecken DParis CSidner C(2006)Towards intelligent QA interfacesProceedings of the 11th international conference on Intelligent user interfaces10.1145/1111449.1111487(163-170)Online publication date: 29-Jan-2006
https://dl.acm.org/doi/10.1145/1111449.1111487
Jörgensen P(2005)Incorporating context in text analysis by interactive activation with competition artificial neural networksInformation Processing & Management10.1016/j.ipm.2004.10.00341:5(1081-1099)Online publication date: Sep-2005
https://doi.org/10.1016/j.ipm.2004.10.003

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Abstract

References

Cited By

Index Terms

Recommendations

Quality-aware collaborative question answering: methods and evaluation

Question Answering System Based on Web

Interactive natural language question answering over knowledge graphs

Comments

Information

Published In

Sponsors

Publisher

Publication History

Permissions

Check for updates

Author Tags

Qualifiers

Conference

Acceptance Rates

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Login options

Full Access

View options

PDF

eReader

Share

Share this Publication link

Share on social media

Affiliations