Systematic Searching in a Post-Pandemic World: New Directions for Methods, Technology, and People

Aria, M., & Cuccurullo, C. (2017). bibliometrix: An R-tool for comprehensive science mapping analysis. Journal of Informetrics, 11(4), 959–975. https://doi.org/10.1016/j.joi.2017.08.007 DOI: https://doi.org/10.1016/j.joi.2017.08.007

Arno, A., Elliott, J., Wallace, B., Turner, T., & Thomas, J. (2021). The views of health guideline developers on the use of automation in health evidence synthesis. Systematic Reviews, 10(1), 16. https://doi.org/10.1186/s13643-020-01569-2 DOI: https://doi.org/10.1186/s13643-020-01569-2

Ashiq, M., & Warraich, N. F. (2022). A systematized review on data librarianship literature: Current services, challenges, skills, and motivational factors. Journal of Librarianship and Information Science, 55(2), 414–433. https://doi.org/10.1177/09610006221083675 DOI: https://doi.org/10.1177/09610006221083675

Beller, E., Clark, J., Tsafnat, G., Adams, C., Diehl, H., Lund, H., Ouzzani, M., Thayer, K., Thomas, J., Turner, T., Xia, J., Robinson, K., & Glasziou, P. (2018). Making progress with the automation of systematic reviews: Principles of the International Collaboration for the Automation of Systematic Reviews (ICASR). Systematic Reviews, 7(1), 77. https://doi.org/10.1186/s13643-018-0740-7 DOI: https://doi.org/10.1186/s13643-018-0740-7

Brierley, L., Nanni, F., Polka, J. K., Dey, G., Pálfy, M., Fraser, N., & Coates, J. A. (2022). Tracking changes between preprint posting and journal publication during a pandemic. PLOS Biology, 20(2), e3001285. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8806067/ DOI: https://doi.org/10.1371/journal.pbio.3001285

Briscoe, S., Abbott, R., & Melendez‐Torres, G. J. (2022). Expert searchers identified time, team, technology and tension as challenges when carrying out supplementary searches for systematic reviews: A thematic network analysis. Health Information & Libraries Journal. Advance online publication. https://doi.org/10.1111/hir.12468 DOI: https://doi.org/10.1111/hir.12468

Brody, S., Loree, S., Sampson, M., Mensinkai, S., Coffman, J., Mueller, M., Askin, N., Hamill, C., Wilson, E., McAteer, M. B., & Staines, H. (2023). Searching for evidence in public health emergencies: A white paper of best practices. Journal of the Medical Library Association, 111(1), 566–578. https://doi.org/10.5195/jmla.2023.1530 DOI: https://doi.org/10.5195/jmla.2023.1530

Butcher, R., Sampson, M., Couban, R. J., Malin, J. E., Loree, S., & Brody, S. (2022). The currency and completeness of specialized databases of COVID-19 publications. Journal of Clinical Epidemiology, 147, 52–59. https://doi.org/10.1016/j.jclinepi.2022.03.006 DOI: https://doi.org/10.1016/j.jclinepi.2022.03.006

CABI Digital Library. (2023). searchRxiv. Retrieved October 12, 2023, from https://www.cabidigitallibrary.org/journal/searchrxiv

Callaway, J. (2021). The Librarian Reserve Corps: An emergency response. Medical Reference Services Quarterly, 40(1), 90–102. https://doi.org/10.1080/02763869.2021.1873627 DOI: https://doi.org/10.1080/02763869.2021.1873627

Chappell, M., Edwards, M., Watkins, D., Marshall, C., & Graziadio, S. (2023). Machine learning for accelerating screening in evidence reviews. Cochrane Evidence Synthesis and Methods, 1(5), e12021. https://doi.org/10.1002/cesm.12021 DOI: https://doi.org/10.1002/cesm.12021

Chen, Y. Y., Bullard, J., & Giustini, D. (2023). Automated indexing using NLM's Medical Text Indexer (MTI) compared to human indexing in Medline: A pilot study. Journal of the Medical Library Association, 111(3), 684–695. https://doi.org/10.5195/jmla.2023.1588 DOI: https://doi.org/10.5195/jmla.2023.1588

Clyne, B., Walsh, K. A., O'Murchu, E., Sharp, M. K., Comber, L., O’ Brien, K. K., Smith, S. M., Harrington, P., O'Neill, M., Teljeur, C., & Ryan, M. (2021). Using preprints in evidence synthesis: Commentary on experience during the COVID-19 pandemic. Journal of Clinical Epidemiology, 138, 203–210. https://doi.org/10.1016/j.jclinepi.2021.05.010 DOI: https://doi.org/10.1016/j.jclinepi.2021.05.010

Cooper, C., Booth, A., Husk, K., Lovell, R., Frost, J., Schauberger, U., Britten, N., & Garside, R. (2022). A Tailored Approach: A model for literature searching in complex systematic reviews. Journal of Information Science. Advance online publication. https://doi.org/10.1177/01655515221114452 DOI: https://doi.org/10.1177/01655515221114452

De Brún, C. (2022, June 1–3). Knowledge makes the world go round: Librarians working together to fight the COVID infodemic [Poster session]. European Association for Health Information and Libraries, Rotterdam, Netherlands.

de Kock, S., Stirk, L., Ross, J., Duffy, S., Noake, C., & Misso, K. (2020). Systematic review search methods evaluated using the Preferred Reporting of Items for Systematic Reviews and Meta-Analyses and the Risk of Bias in Systematic reviews tool. International Journal of Technology Assessment in Health Care, 37(1), E18. https://doi.org/10.1017/S0266462320002135 DOI: https://doi.org/10.1017/S0266462320002135

EBI-SIG. (2023). Library of Search Strategy Resources. Retrieved October 12, 2023, from https://sites.google.com/view/searchresourceslib/home

El Mikati, I. K., Khabsa, J., Harb, T., Khamis, M., Agarwal, A., Pardo-Hernandez, H., Farran, S., Khamis, A. M., El Zein, O., El-Khoury, R., Schünemann, H. J., & Akl, E. A. (2022). A Framework for the development of living practice guidelines in health care. Annals of Internal Medicine, 175(8), 1154-1160. https://doi.org/10.7326/M22-0514 DOI: https://doi.org/10.7326/M22-0514

Elliott, J. H., Synnot, A., Turner, T., Simmonds, M., Akl, E. A., McDonald, S., Salanti, G., Meerpohl, J., MacLehose, H., Hilton, J., Tovey, D., Shemilt, I., & Thomas, J. (2017). Living systematic review: 1. Introduction—the why, what, when, and how. Journal of Clinical Epidemiology, 91, 23–30. https://doi.org/10.1016/j.jclinepi.2017.08.010 DOI: https://doi.org/10.1016/j.jclinepi.2017.08.010

Foster, M. J., & Jewell, S. T. (2022). Piecing together systematic reviews and other evidence syntheses. Rowman & Littlefield. DOI: https://doi.org/10.5771/9781538150184

Greenhalgh, T., Fisman, D., Cane, D. J., Oliver, M., & Macintyre, C. R. (2022). Adapt or die: How the pandemic made the shift from EBM to EBM+ more urgent. BMJ Evidence-Based Medicine, 27(5), 253–260. https://doi.org/10.1136/bmjebm-2022-111952 DOI: https://doi.org/10.1136/bmjebm-2022-111952

Greenhalgh, T., & Peacock, R. (2005). Effectiveness and efficiency of search methods in systematic reviews of complex evidence: Audit of primary sources. BMJ, 331(7524), 1064–1065. https://doi.org/10.1136/bmj.38636.593461.68 DOI: https://doi.org/10.1136/bmj.38636.593461.68

Johnson, E. E., O’Keefe, H., Sutton, A., & Marshall, C. (2022). The Systematic Review Toolbox: Keeping up to date with tools to support evidence synthesis. Systematic Reviews, 11(1), 258. https://doi.org/10.1186/s13643-022-02122-z DOI: https://doi.org/10.1186/s13643-022-02122-z

Khalil, H., Tamara, L., Rada, G., & Akl, E. A. (2021). Challenges of evidence synthesis during the 2020 COVID pandemic: A scoping review. Journal of Clinical Epidemiology, 142, 10–18. https://doi.org/10.1016/J.JCLINEPI.2021.10.017 DOI: https://doi.org/10.1016/j.jclinepi.2021.10.017

Kirkham, J. J., Penfold, N. C., Murphy, F., Boutron, I., Ioannidis, J. P., Polka, J., & Moher, D. (2020). Systematic examination of preprint platforms for use in the medical and biomedical sciences setting. BMJ Open, 10(12), e041849. https://doi.org/10.1136/bmjopen-2020-041849 DOI: https://doi.org/10.1136/bmjopen-2020-041849

Levay, P., & Craven, J. (2019). Conclusion: Where do we go from here? In P. Levay, & J. Craven (Eds.), Systematic searching: Practical ideas for improving results (pp. 289–292). Facet Publishing. DOI: https://doi.org/10.29085/9781783303755.015

Levay, P., Heath, A., & Tuvey, D. (2022a). Efficient searching for NICE public health guidelines: Would using fewer sources still find the evidence? Research Synthesis Methods, 13(6), 760–789. https://doi.org/10.1002/jrsm.1577 DOI: https://doi.org/10.1002/jrsm.1577

Levay, P., Walsh, N., & Foster, L. (2022b). The National Institute for Health and Care Excellence information specialist development pathway: Developing the skills, knowledge and confidence to quality assure search strategies. Health Information and Libraries Journal, 39(4), 392–399. https://doi.org/10.1111/hir.12460 DOI: https://doi.org/10.1111/hir.12460

MacFarlane, A., Russell-Rose, T., & Shokraneh, F. (2022). Search strategy formulation for systematic reviews: Issues, challenges and opportunities. Intelligent Systems with Applications, 15, 200091. https://doi.org/10.1016/j.iswa.2022.200091 DOI: https://doi.org/10.1016/j.iswa.2022.200091

McDonald, S., Sharp, S., Morgan, R. L., Murad, M. H., & Fraile Navarro, D. (2023). Methods for living guidelines: Early guidance based on practical experience. Paper 4: Search methods and approaches for living guidelines. Journal of Clinical Epidemiology, 155, 108–117. https://doi.org/10.1016/j.jclinepi.2022.12.023 DOI: https://doi.org/10.1016/j.jclinepi.2022.12.023

Metzendorf, M., & Featherstone, R. M. (2021). Evaluation of the comprehensiveness, accuracy and currency of the Cochrane COVID-19 Study Register for supporting rapid evidence synthesis production. Research Synthesis Methods, 12(5), 607–617. https://doi.org/10.1002/jrsm.1501 DOI: https://doi.org/10.1002/jrsm.1501

Metzendorf, M., Weibel, S., Reis, S., & McDonald, S. (2022). Pragmatic and open science-based solution to a current problem in the reporting of living systematic reviews. BMJ Evidence-Based Medicine, 8(4), 267–272. https://doi.org/10.1136/bmjebm-2022-112019 DOI: https://doi.org/10.1136/bmjebm-2022-112019

Naicker, R. (2022). Critically appraising for antiracism. Education for Information, 38(4), 291–308. https://doi.org/10.3233/EFI-220052 DOI: https://doi.org/10.3233/EFI-220052

National Library of Medicine. (2022). Indexing FAQs. Retrieved October 12, 2023, from https://support.nlm.nih.gov/knowledgebase/article/KA-05326/en-us

O’Mara-Eves, A., Thomas, J., McNaught, J., Miwa, M., & Ananiadou, S. (2015). Using text mining for study identification in systematic reviews: A systematic review of current approaches. Systematic Reviews, 4(1), 5. https://doi.org/10.1186/2046-4053-4-5 DOI: https://doi.org/10.1186/2046-4053-4-5

Pierre, O., Riveros, C., Charpy, S., & Boutron, I. (2021). Secondary electronic sources demonstrated very good sensitivity for identifying studies evaluating interventions for COVID-19. Journal of Clinical Epidemiology, 141, 46–53. https://doi.org/10.1016/j.jclinepi.2021.09.022 DOI: https://doi.org/10.1016/j.jclinepi.2021.09.022

Qureshi, R., Shaughnessy, D., Gill, K. A. R., Robinson, K. A., Li, T., & Agai, E. (2023). Are ChatGPT and large language models “the answer” to bringing us closer to systematic review automation? Systematic Reviews, 12(1), 72. https://doi.org/10.1186/s13643-023-02243-z DOI: https://doi.org/10.1186/s13643-023-02243-z

Ramirez, D., Foster, M. J., Kogut, A., & Xiao, D. (2022). Adherence to systematic review standards: Impact of librarian involvement in Campbell Collaboration's education reviews. The Journal of Academic Librarianship, 48(5), 102567. https://doi.org/10.1016/j.acalib.2022.102567 DOI: https://doi.org/10.1016/j.acalib.2022.102567

Rethlefsen, M. L., Kirtley, S., Waffenschmidt, S., Ayala, A. P., Moher, D., Page, M. J., & Koffel, J. B. (2021). PRISMA-S: An extension to the PRISMA statement for reporting literature searches in systematic reviews. Systematic Reviews, 10(1), 39. https://doi.org/10.1186/s13643-020-01542-z DOI: https://doi.org/10.1186/s13643-020-01542-z

Rosonovski, S., Levchenko, M., Ide‐Smith, M., Faulk, L., Harrison, M., & McEntyre, J. (2023). Searching and evaluating publications and preprints using Europe PMC. Current Protocols, 3(3), e694. https://doi.org/10.1002/cpz1.694 DOI: https://doi.org/10.1002/cpz1.694

Sampson, M. (2019). Communication for information specialists. In P. Levay, & J. Craven (Eds.), Systematic searching: Practical ideas for improving results (pp. 249–268). Facet Publishing. DOI: https://doi.org/10.29085/9781783303755.013

Shemilt, I., Noel-Storr, A., Thomas, J., Featherstone, R., & Mavergames, C. (2022). Machine learning reduced workload for the Cochrane COVID-19 Study Register: Development and evaluation of the Cochrane COVID-19 Study Classifier. Systematic Reviews, 11(1), 15. https://doi.org/10.1186/s13643-021-01880-6 DOI: https://doi.org/10.1186/s13643-021-01880-6

Shokraneh, F., & Adams, C. E. (2019). Study-based registers reduce waste in systematic reviewing: Discussion and case report. Systematic Reviews, 8(1), 129. https://doi.org/10.1186/s13643-019-1035-3 DOI: https://doi.org/10.1186/s13643-019-1035-3

Stansfield, C., Stokes, G., & Thomas, J. (2022). Applying machine classifiers to update searches: Analysis from two case studies. Research Synthesis Methods, 13(1), 121–133. https://doi.org/10.1002/jrsm.1537 DOI: https://doi.org/10.1002/jrsm.1537

Svarre, T., & Russell-Rose, T. (2022). Think outside the search box: A comparative study of visual and form-based query builders. Journal of Information Science. Advance online publication. https://doi.org/10.1177/01655515221138536 DOI: https://doi.org/10.1177/01655515221138536

Thomas, J., McDonald, S., Noel-Storr, A., Shemilt, I., Elliott, J., Mavergames, C., & Marshall, I. J. (2021). Machine learning reduced workload with minimal risk of missing studies: Development and evaluation of a randomized controlled trial classifier for Cochrane Reviews. Journal of Clinical Epidemiology, 133, 140–151. https://doi.org/10.1016/j.jclinepi.2020.11.003 DOI: https://doi.org/10.1016/j.jclinepi.2020.11.003

Townsend, W., Anderson, P., Haines, K., Hansen, S., James, L., MacEachern, M., Rana, G., Saylor, K. & Saylor, K. (2022). Addressing antiquated, non-standard, exclusionary, and potentially offensive terms in evidence syntheses and systematic searches. Retrieved October 12, 2023, from https://doi.org/10.7302/6408

van Noorden, R. (2023). ChatGPT-like AIs are coming to major science search engines. Nature, 620(7973), 258. https://doi.org/10.1016/j.jclinepi.2022.12.023 DOI: https://doi.org/10.1038/d41586-023-02470-3

Verdugo-Paiva, F., Vergara, C., Ávila, C., Castro-Guevara, J., Cid, J., Contreras, V., Jara, I., Jiménez, V., Lee, M. H., Muñoz, M., Rojas-Gómez, A. M., Rosón-Rodríguez, P., Serrano-Arévalo, K., Silva-Ruz, I., Vásquez-Laval, J., Zambrano-Achig, P., Zavadzki, G., & Rada, G. (2022). COVID-19 Living Overview of Evidence repository is highly comprehensive and can be used as a single source for COVID-19 studies. Journal of Clinical Epidemiology, 149, 195–202. https://doi.org/10.1016/j.jclinepi.2022.05.001 DOI: https://doi.org/10.1016/j.jclinepi.2022.05.001

Waffenschmidt, S., & Hausner, E. (2019). Collaborative working to improve searching. In P. Levay, & J. Craven (Eds.), Systematic searching: Practical ideas for improving results (pp. 229–248). Facet Publishing. DOI: https://doi.org/10.29085/9781783303755.012

Wang, S., Scells, H., Koopman, B., & Zuccon, G. (2023). Can ChatGPT write a good Boolean query for systematic review literature search? SIGIR '23: Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval, Taipei. 1426–1436. https://doi.org/10.1145/3539618.3591703 DOI: https://doi.org/10.1145/3539618.3591703