skip to main content
10.1145/3209978.3210020acmconferencesArticle/Chapter ViewAbstractPublication PagesirConference Proceedingsconference-collections
research-article

Generating Better Queries for Systematic Reviews

Authors Info & Claims
Published:27 June 2018Publication History

ABSTRACT

Systematic reviews form the cornerstone of evidence based medicine, aiming to answer complex medical questions based on all evidence currently available. Key to the effectiveness of a systematic review is an (often large) Boolean query used to search large publication repositories. These Boolean queries are carefully crafted by researchers and information specialists, and often reviewed by a panel of experts. However, little is known about the effectiveness of the Boolean queries at the time of formulation. In this paper we investigate whether a better Boolean query than that defined in the protocol of a systematic review, can be created, and we develop methods for the transformation of a given Boolean query into a more effective one. Our approach involves defining possible transformations of Boolean queries and their clauses. It also involves casting the problem of identifying a transformed query that is better than the original into: (i) a classification problem; and (ii) a learning to rank problem. Empirical experiments are conducted on a real set of systematic reviews. Analysis of results shows that query transformations that are better than the original queries do exist, and that our approaches are able to select more effective queries from the set of possible transformed queries so as to maximise different target effectiveness measures.

References

  1. T. Agoritsas, A. Merglen, D.S. Courvoisier, C. Combescure, N. Garin, A. Perrier, and T.V. Perneger . 2012. Sensitivity and predictive value of 15 PubMed search strategies to answer clinical questions rated against full systematic reviews. JMIR Vol. 14, 3 (2012), e85.Google ScholarGoogle Scholar
  2. N. Balasubramanian, G. Kumaran, and V.R. Carvalho . 2010. Exploring reductions for long web queries. In SIGIR'10. ACM, 571--578. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. C.C. Chang and C.J. Lin . 2011. LIBSVM: A library for support vector machines. TIST Vol. 2 (2011), 27:1--27:27. Issue 3. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. A.M. Cohen, W.R. Hersh, K. Peterson, and P.Y. Yen . 2006. Reducing workload in systematic review preparation using automated citation classification. JAMIA Vol. 13, 2 (2006), 206--219.Google ScholarGoogle Scholar
  5. S. Golder, Y. Loke, and H. M. McIntosh . 2008. Poor reporting and inadequate searches were apparent in systematic reviews of adverse effects. Journal of Clinical Epidemiology (2008).Google ScholarGoogle Scholar
  6. T. Greenhalgh and R. Peacock . 2005. Effectiveness and efficiency of search methods in systematic reviews of complex evidence: audit of primary sources. BMJ Vol. 331, 7524 (2005), 1064--1065.Google ScholarGoogle Scholar
  7. T. Joachims . 2006. Training linear SVMs in linear time. In SIGKDD'06. 217--226. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Green S JPT CHH . 2011. Cochrane handbook for systematic reviews of interventions version 5.1.0 {updated March 2011}. The Cochrane Collaboration (2011).Google ScholarGoogle Scholar
  9. E. Kanoulas, D. Li, L. Azzopardi, and R. Spijker . 2017. CLEF 2017 Technologically Assisted Reviews in Empirical Medicine Overview CLEF'17.Google ScholarGoogle Scholar
  10. S. Karimi, S. Pohl, F. Scholer, L. Cavedon, and J. Zobel . 2010. Boolean versus ranked querying for biomedical systematic reviews. BMC MIDM Vol. 10, 1 (2010), 1.Google ScholarGoogle Scholar
  11. M. Khabsa, A. Elmagarmid, I. Ilyas, H. Hammady, and M. Ouzzani . 2016. Learning to identify relevant studies for systematic reviews using random forest and external information. ML Vol. 102, 3 (2016), 465--482. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. S.N. Kim, D. Martinez, L. Cavedon, and L. Yencken . 2011 a. Automatic classification of sentences to support evidence based medicine. BMC bioinformatics Vol. 12, 2 (2011).Google ScholarGoogle Scholar
  13. Y. Kim, J. Seo, and W. B. Croft . 2011 b. Automatic boolean query suggestion for professional search SIGIR'11. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. G. Kumaran and V.R. Carvalho . 2009. Reducing long queries using query quality predictors SIGIR'09. 564--571. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. J. Lavis, H. Davies, A. Oxman, J.L. Denis, K. Golden-Biddle, and E. Ferlie . 2005. Towards systematic reviews that inform health care management and policy-making. JHSRP Vol. 10, 1_suppl (2005), 35--48.Google ScholarGoogle Scholar
  16. G. Luo, C. Tang, H. Yang, and X. Wei . 2008. MedSearch: a specialized search engine for medical information retrieval CIKM'08. 143--152. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. D. Martinez, S. Karimi, L. Cavedon, and T. Baldwin . 2008. Facilitating biomedical systematic reviews using ranked text retrieval and classification. In ADCD'08.Google ScholarGoogle Scholar
  18. M. Miwa, J. Thomas, A. O'Mara-Eves, and S. Ananiadou . 2014. Reducing systematic review workload through certainty-based screening. JBI Vol. 51 (2014), 242--253. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. M. Sampson and J. McGowan . 2006. Errors in search strategies were identified by type and frequency. JCE Vol. 59, 10 (2006), 1057--e1.Google ScholarGoogle Scholar
  20. H. Scells, L. Azzopardi, G. Zuccon, and B. Koopman . 2018 a. Query Variation Performance Prediction for Systematic Reviews SIGIR'18. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. H. Scells, D. Locke, and G. Zuccon . 2018 b. An Information Retrieval Experiment Framework for Domain Specific Applications SIGIR'18. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. H. Scells, G. Zuccon, B. Koopman, A. Deacon, L. Azzopardi, and S. Geva . 2017 a. Integrating the framing of clinical questions via PICO into the retrieval of medical literature for systematic reviews. In CIKM'17. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. H. Scells, G. Zuccon, B. Koopman, A. Deacon, S. Geva, and L. Azzopardi . 2017 b. A Test Collection for Evaluating Retrieval of Studies for Inclusion in Systematic Reviews. In SIGIR'2017. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. I. Shemilt, A. Simon, G.J. Hollands, T.M. Marteau, D. Ogilvie, A. O'Mara-Eves, M.P. Kelly, and J. Thomas . 2014. Pinpointing needles in giant haystacks: use of text mining to reduce impractical screening workload in extremely large scoping reviews. RSM Vol. 5, 1 (2014), 31--49.Google ScholarGoogle Scholar
  25. L. Soldaini, A. Cohan, A. Yates, N. Goharian, and O. Frieder . 2015. Retrieving medical literature for clinical decision support ECIR'15. 538--549.Google ScholarGoogle Scholar
  26. G. Tsafnat, P. Glasziou, M.K. Choong, A. Dunn, F. Galgani, and E. Coiera . 2014. Systematic review automation technologies. SR Vol. 3, 1 (2014), 74.Google ScholarGoogle Scholar
  27. B.C. Wallace, T.A. Trikalinos, J. Lau, C. Brodley, and C.H. Schmid . 2010. Semi-automated screening of biomedical citations for systematic reviews. BMC Bio Vol. 11, 1 (Jan . 2010), 55.Google ScholarGoogle ScholarCross RefCross Ref
  28. A. Yoshii, D.A. Plaut, K.A. McGraw, M.J. Anderson, and K.E. Wellik . 2009. Analysis of the reporting of search strategies in Cochrane systematic reviews. JMLA Vol. 97, 1 (2009), 21.Google ScholarGoogle ScholarCross RefCross Ref
  29. G. Zuccon, B. Koopman, A. Nguyen, D. Vickers, and L. Butt . 2012. Exploiting Medical Hierarchies for Concept-based Information Retrieval ADCS'12. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Generating Better Queries for Systematic Reviews

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in
    • Published in

      cover image ACM Conferences
      SIGIR '18: The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval
      June 2018
      1509 pages
      ISBN:9781450356572
      DOI:10.1145/3209978

      Copyright © 2018 ACM

      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 27 June 2018

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • research-article

      Acceptance Rates

      SIGIR '18 Paper Acceptance Rate86of409submissions,21%Overall Acceptance Rate792of3,983submissions,20%

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader