ABSTRACT
Systematic reviews form the cornerstone of evidence based medicine, aiming to answer complex medical questions based on all evidence currently available. Key to the effectiveness of a systematic review is an (often large) Boolean query used to search large publication repositories. These Boolean queries are carefully crafted by researchers and information specialists, and often reviewed by a panel of experts. However, little is known about the effectiveness of the Boolean queries at the time of formulation. In this paper we investigate whether a better Boolean query than that defined in the protocol of a systematic review, can be created, and we develop methods for the transformation of a given Boolean query into a more effective one. Our approach involves defining possible transformations of Boolean queries and their clauses. It also involves casting the problem of identifying a transformed query that is better than the original into: (i) a classification problem; and (ii) a learning to rank problem. Empirical experiments are conducted on a real set of systematic reviews. Analysis of results shows that query transformations that are better than the original queries do exist, and that our approaches are able to select more effective queries from the set of possible transformed queries so as to maximise different target effectiveness measures.
- T. Agoritsas, A. Merglen, D.S. Courvoisier, C. Combescure, N. Garin, A. Perrier, and T.V. Perneger . 2012. Sensitivity and predictive value of 15 PubMed search strategies to answer clinical questions rated against full systematic reviews. JMIR Vol. 14, 3 (2012), e85.Google Scholar
- N. Balasubramanian, G. Kumaran, and V.R. Carvalho . 2010. Exploring reductions for long web queries. In SIGIR'10. ACM, 571--578. Google ScholarDigital Library
- C.C. Chang and C.J. Lin . 2011. LIBSVM: A library for support vector machines. TIST Vol. 2 (2011), 27:1--27:27. Issue 3. Google ScholarDigital Library
- A.M. Cohen, W.R. Hersh, K. Peterson, and P.Y. Yen . 2006. Reducing workload in systematic review preparation using automated citation classification. JAMIA Vol. 13, 2 (2006), 206--219.Google Scholar
- S. Golder, Y. Loke, and H. M. McIntosh . 2008. Poor reporting and inadequate searches were apparent in systematic reviews of adverse effects. Journal of Clinical Epidemiology (2008).Google Scholar
- T. Greenhalgh and R. Peacock . 2005. Effectiveness and efficiency of search methods in systematic reviews of complex evidence: audit of primary sources. BMJ Vol. 331, 7524 (2005), 1064--1065.Google Scholar
- T. Joachims . 2006. Training linear SVMs in linear time. In SIGKDD'06. 217--226. Google ScholarDigital Library
- Green S JPT CHH . 2011. Cochrane handbook for systematic reviews of interventions version 5.1.0 {updated March 2011}. The Cochrane Collaboration (2011).Google Scholar
- E. Kanoulas, D. Li, L. Azzopardi, and R. Spijker . 2017. CLEF 2017 Technologically Assisted Reviews in Empirical Medicine Overview CLEF'17.Google Scholar
- S. Karimi, S. Pohl, F. Scholer, L. Cavedon, and J. Zobel . 2010. Boolean versus ranked querying for biomedical systematic reviews. BMC MIDM Vol. 10, 1 (2010), 1.Google Scholar
- M. Khabsa, A. Elmagarmid, I. Ilyas, H. Hammady, and M. Ouzzani . 2016. Learning to identify relevant studies for systematic reviews using random forest and external information. ML Vol. 102, 3 (2016), 465--482. Google ScholarDigital Library
- S.N. Kim, D. Martinez, L. Cavedon, and L. Yencken . 2011 a. Automatic classification of sentences to support evidence based medicine. BMC bioinformatics Vol. 12, 2 (2011).Google Scholar
- Y. Kim, J. Seo, and W. B. Croft . 2011 b. Automatic boolean query suggestion for professional search SIGIR'11. Google ScholarDigital Library
- G. Kumaran and V.R. Carvalho . 2009. Reducing long queries using query quality predictors SIGIR'09. 564--571. Google ScholarDigital Library
- J. Lavis, H. Davies, A. Oxman, J.L. Denis, K. Golden-Biddle, and E. Ferlie . 2005. Towards systematic reviews that inform health care management and policy-making. JHSRP Vol. 10, 1_suppl (2005), 35--48.Google Scholar
- G. Luo, C. Tang, H. Yang, and X. Wei . 2008. MedSearch: a specialized search engine for medical information retrieval CIKM'08. 143--152. Google ScholarDigital Library
- D. Martinez, S. Karimi, L. Cavedon, and T. Baldwin . 2008. Facilitating biomedical systematic reviews using ranked text retrieval and classification. In ADCD'08.Google Scholar
- M. Miwa, J. Thomas, A. O'Mara-Eves, and S. Ananiadou . 2014. Reducing systematic review workload through certainty-based screening. JBI Vol. 51 (2014), 242--253. Google ScholarDigital Library
- M. Sampson and J. McGowan . 2006. Errors in search strategies were identified by type and frequency. JCE Vol. 59, 10 (2006), 1057--e1.Google Scholar
- H. Scells, L. Azzopardi, G. Zuccon, and B. Koopman . 2018 a. Query Variation Performance Prediction for Systematic Reviews SIGIR'18. Google ScholarDigital Library
- H. Scells, D. Locke, and G. Zuccon . 2018 b. An Information Retrieval Experiment Framework for Domain Specific Applications SIGIR'18. Google ScholarDigital Library
- H. Scells, G. Zuccon, B. Koopman, A. Deacon, L. Azzopardi, and S. Geva . 2017 a. Integrating the framing of clinical questions via PICO into the retrieval of medical literature for systematic reviews. In CIKM'17. Google ScholarDigital Library
- H. Scells, G. Zuccon, B. Koopman, A. Deacon, S. Geva, and L. Azzopardi . 2017 b. A Test Collection for Evaluating Retrieval of Studies for Inclusion in Systematic Reviews. In SIGIR'2017. Google ScholarDigital Library
- I. Shemilt, A. Simon, G.J. Hollands, T.M. Marteau, D. Ogilvie, A. O'Mara-Eves, M.P. Kelly, and J. Thomas . 2014. Pinpointing needles in giant haystacks: use of text mining to reduce impractical screening workload in extremely large scoping reviews. RSM Vol. 5, 1 (2014), 31--49.Google Scholar
- L. Soldaini, A. Cohan, A. Yates, N. Goharian, and O. Frieder . 2015. Retrieving medical literature for clinical decision support ECIR'15. 538--549.Google Scholar
- G. Tsafnat, P. Glasziou, M.K. Choong, A. Dunn, F. Galgani, and E. Coiera . 2014. Systematic review automation technologies. SR Vol. 3, 1 (2014), 74.Google Scholar
- B.C. Wallace, T.A. Trikalinos, J. Lau, C. Brodley, and C.H. Schmid . 2010. Semi-automated screening of biomedical citations for systematic reviews. BMC Bio Vol. 11, 1 (Jan . 2010), 55.Google ScholarCross Ref
- A. Yoshii, D.A. Plaut, K.A. McGraw, M.J. Anderson, and K.E. Wellik . 2009. Analysis of the reporting of search strategies in Cochrane systematic reviews. JMLA Vol. 97, 1 (2009), 21.Google ScholarCross Ref
- G. Zuccon, B. Koopman, A. Nguyen, D. Vickers, and L. Butt . 2012. Exploiting Medical Hierarchies for Concept-based Information Retrieval ADCS'12. Google ScholarDigital Library
Index Terms
Generating Better Queries for Systematic Reviews
Recommendations
Automatic Boolean Query Refinement for Systematic Review Literature Search
WWW '19: The World Wide Web ConferenceIn the medical domain, systematic reviews are a highly trustworthy evidence source used to inform clinical diagnosis and treatment, and governmental policy making. Systematic reviews must be complete in that all relevant literature for the research ...
A comparison of automatic Boolean query formulation for systematic reviews
AbstractSystematic reviews are comprehensive literature reviews that target a highly focused research question. In the medical domain, complex Boolean queries are used to identify studies. To ensure comprehensiveness, all studies retrieved are screened ...
Smooth Operators for Effective Systematic Review Queries
SIGIR '23: Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information RetrievalEffective queries are crucial to minimising the time and cost of medical systematic reviews, as all retrieved documents must be judged for relevance. Boolean queries, developed by expert librarians, are the standard for systematic reviews. They guarantee ...
Comments