Examining the Meaning and Methodological Characteristics of the Systematized Review Label: A Scoping Review
DOI:
https://doi.org/10.18438/eblip30757Abstract
Objective – In 2009, a typology by Grant and Booth introduced the concept of a systematized review in which authors (typically students) selectively employ various elements of the systematic review process. As academic librarians who help research teams to select and conduct a variety of evidence synthesis types, we have fielded inquiries from teams interested in conducting systematized reviews for publication. While the typology is widely referenced, we were aware of no formal methodological guide to a systematized review type.
The objective of this scoping review is to identify and describe the extent of published systematized reviews, and to 1) identify and collate, where available, sources used for the conceptualization and conduct of the systematized reviews, 2) determine if explanations provided were based on constraints, and 3) describe common methodological characteristics.
Methods – Articles titled as a systematized review that attempted the collocation and synthesis of literature and included an adequate description of their methodology were included. We searched on September 1, 2023, the following five sources: Google Scholar, Lens.org, Web of Science Core Collection, Scopus, and MEDLINE. We performed screening and data extraction in duplicate. Data extraction elements included common methodological characteristics relating to various steps of the evidence synthesis process. Descriptive, aggregate statistics, and categorization of reasons for selecting the systematized review type were the primary analysis for this review.
Results – Our review found 171 systematized reviews that met inclusion criteria. These were published between 2013 and 2023, and the number has increased each year. Resources or constraints were mentioned in only 32 included reviews, and only 19 had a single author. The methodological attributes of published systematized reviews vary significantly. A small number (15) of reviews searched only one database or source, while the majority searched between 2 and 6 sources. The majority (134) provided no search details or a non-reproducible search strategy. Only 36 included reviews mentioned librarian involvement.
Conclusion – Librarians should stay abreast of emerging evidence synthesis practices and the context in which researchers operate. It is unclear why and how authors choose the systematized review type because most did not report a source for their conceptualization, nor justify their choice. The majority of published reviews did not connect their methodological departures to the kinds of resource constraints typically associated with student work. The expectation that the elements of the systematic review process that authors chose to incorporate should be conducted and reported according to standards was not widely met.
Librarians, as methodological guides, participants, or evaluators can play a key role in reinforcing the expectations of this review type and the standards that should be met when publishing a systematized review. Publishers and consumers of systematized reviews are advised to consider how each modification of standard systematic review methodology may introduce bias and affect the value of the evidence presented in published systematized reviews.
Downloads
References
Amog, K., Pham, B., Courvoisier, M., Mak, M., Booth, A., Godfrey, C., Hwee, J., Straus, S. E., & Tricco, A. C. (2022). The web-based “Right Review” tool asks reviewers simple questions to suggest methods from 41 knowledge synthesis methods. Journal of Clinical Epidemiology, 147, 42–51. https://doi.org/10.1016/j.jclinepi.2022.03.004
Aromataris, E., Lockwood, C., Porritt, K., Pilla, B., & Jordan, Z. (Eds.). (2024). JBI manual for evidence synthesis. JBI. https://doi.org/10.46658/JBIMES-24-01
Borrego, M., Foster, M. J., & Froyd, J. E. (2014). Systematic literature reviews in engineering education and other developing interdisciplinary fields. Journal of Engineering Education, 103(1), 45–76. https://doi.org/10.1002/jee.20038
Codina, L. (2020). Revisiones sistematizadas en ciencias humanas y sociales. 3: Análisis y Síntesis de la información cualitativa. In C. Lopezosa, J. Díaz-Noci, & Ll. Codina, (Eds.), Methodos anuario de métodos de investigación en comunicación social, 1 (pp. 73–87). Universitat Pompeu Fabra. https://doi.org/10.31009/methodos.2020.i01.07
Codina, L. (2018a). Revisiones bibliográficas sistematizadas: Procedimientos generales y framework para ciencias humanas y sociales [Master's Thesis, Universitat Pompeu Fabra]. e-Repositori upf. http://hdl.handle.net/10230/34497
Codina, L. (2018b). Sistemas de búsqueda y obtención de información: Componentes y evolución. Anuario ThinkEPI, 12, 77–82. https://doi.org/10.3145/thinkepi.2018.06
Grant, M. J., & Booth, A. (2009). A typology of reviews: An analysis of 14 review types and associated methodologies. Health Information & Libraries Journal, 26(2), 91–108. https://doi.org/10.1111/j.1471-1842.2009.00848.x
Higgins, J. P. T., Thomas, J., Chandler, J., Cumpston, M., Li, T., Page, M. J., & Welch, V. A. (Eds.). (2024). Cochrane handbook for systematic reviews of interventions (Version 6.5). Cochrane. www.training.cochrane.org/handbook
Khalil, H., Campbell, F., Danial, K., Pollock, D., Munn, Z., Welsh, V., Saran, A., Hoppe, D., & Tricco, A. C. (2024). Advancing the methodology of mapping reviews: A scoping review. Research Synthesis Methods, 15(3), 384–397. https://doi.org/10.1002/jrsm.1694
Lefebvre, C., Glanville, J., Featherstone, R., Littlewood, A., & Metzendorf, M. I. (2024). Chapter 4: Searching for and selecting studies. In Higgins, J. P. T., Thomas, J., Chandler, J., Cumpston, M., Li, T., Page, M. J., & Welch, V. A. (Eds.), Cochrane Handbook for Systematic Reviews of Interventions (Version 6.5). Cochrane. https://cochrane.org/handbook
Munn, Z., & Pollock, D. (2024). SPECTRAL Database: Guidance document [PDF]. https://osf.io/vwmfd
Munn, Z., Stern, C., Aromataris, E., Lockwood, C., & Jordan, Z. (2018). What kind of systematic review should I conduct? A proposed typology and guidance for systematic reviewers in the medical and health sciences. BMC Medical Research Methodology, 18, 1-9. https://doi.org/10.1186/s12874-017-0468-4
Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., Chou, R., Glanville, J., Grimshaw, J. M., Hróbjartsson, A., Lalu, M. M., Li, T., Loder, E. W., Mayo-Wilson, E., McDonald, S., … Moher, D. (2021). The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. Systematic Reviews, 10, 1–11. https://doi.org/10.1186/s13643-021-01626-4
Premji, Z., & Cabugos, L. (2023). Examining the meaning and methodological characteristics of the systematized review label: A scoping review protocol. PLOS ONE, 18(9), 1–7. https://doi.org/10.1371/journal.pone.0291145
Peters, M. D. J., Godfrey, C., McInerney, P., Munn, Z., Tricco, A. C., & Khalil, H. (2020). Chapter 10: Scoping reviews. In E. Aromataris & Z. Munn (Eds.), JBI Manual for Evidence Synthesis. JBI. https://doi.org/10.46658/JBIMES-24-09
Shea, B. J., Reeves, B. C., Wells, G., Thuku, M., Hamel, C., Moran, J., Moher, D., Tugwell, P., Welch, V., Kristjansson, E., & Henry, D. A. (2017). AMSTAR 2: A critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ, 358, 1–9. https://doi.org/10.1136/bmj.j4008
Sutton, A., Clowes, M., Preston, L., & Booth, A. (2019). Meeting the review family: Exploring review types and associated information retrieval requirements. Health Information & Libraries Journal, 36(3), 202–222. https://doi.org/10.1111/hir.12276
Tricco, A. C., Lillie, E., Zarin, W., O’Brien, K. K., Colquhoun, H., Levac, D., Moher, D., Peters, M. D. J., Horsley, T., Weeks, L., Hempel, S., Akl, E. A., Chang, C., McGowan, J., Stewart, L., Hartling, L., Aldcroft, A., Wilson, M. G., Garritty, C., … Straus, S. E. (2018). PRISMA extension for scoping reviews (PRISMA-ScR): Checklist and explanation. Annals of Internal Medicine, 169(7), 467–473. https://doi.org/10.7326/M18-0850
Urrútia, G., & Bonfill, X. (2010). Declaración PRISMA: Una propuesta para mejorar la publicación de revisiones sistemáticas y metaanálisis. Medicina Clínica, 135(11), 507–511. https://doi.org/10.1016/j.medcli.2010.01.015
Vrabel, M. (2015). Preferred Reporting Items for Systematic Reviews and Meta-Analyses. Oncology Nursing Forum, 42(5), 552–554. https://doi.org/10.1188/15.ONF.552-554
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Leyla Cabugos, Zahra Premji
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
The Creative Commons-Attribution-Noncommercial-Share Alike License 4.0 International applies to all works published by Evidence Based Library and Information Practice. Authors will retain copyright of the work.
