- Library
- Search Smart
- Research
- Systematic Reviews
- Methodology overview
Systematic Reviews
What's involved?
A systematic review requires:
Time: A systematic review is a rigorous, time-consuming undertaking
A team: You need to work with subject experts, at least one other reviewer for screening citations for relevance, and a librarian who can advise on suitable resources and develop the search strategies, if required. If you are including a meta-analysis, you may also need the help of a statistician. Someone is also required to manage the data and write up the report.
An initial scoping search: This will reveal if a systematic review has already been undertaken on your topic as well as highlight gaps in the knowledge base. A scoping search will also allow you to develop and pilot inclusion/exclusion criteria, and should give you an initial sense of the quantity of existing primary research on your topic (and therefore the scale of the project).
A clearly focused question: This should flow from your initial search. The question should be worthy of an answer and should strike a suitable balance between being too broad or too narrow in scope. If appropriate, use the PICO framework to identify the important concepts in your question.
Clear inclusion/exclusion criteria: These should be based on your question. Consider all aspects of the topic such as age groups, geographic regions, types of study designs, languages, stage of an illness, and the outcome measures that need to be described. Clear eligibility criteria will make it easier to identify relevant articles at the screening stage and prevent you being distracted by interesting but irrelevant studies. Be wary of date range limits in a systematic review. They must be justifiable, not arbitrary.
A comprehensive literature search: This must include a search for unpublished studies as well as published ones. It should target an exhaustive set of relevant databases and include a Google search for grey literature. You might even consider searching the contents pages of all issues of a select set of highly relevant journals. This is called 'hand searching'. Search strategies need to be well documented, including enough detail that they can be replicated. The numbers of results from each database search should also be included.
Critical appraisal: Each study meeting the inclusion criteria must be appraised for quality (internal validity) as well as applicability (external validity or generalisability). A critical appraisal tool such as the CASP RCT checklist can guide you in this.
Good citation management: EndNote is good for taking your search results offline and eventually deduplicating the results from multiple databases. Create a separate EndNote Library for each database searched plus one for citations found via other means (e.g. consulting reference lists or hand searching). Copy all citations into one total library and use the EndNote 'Find Duplicates' command to deduplicate. You will still need to visually remove duplicates, as this process is not perfect. This then becomes your working library.
The systematic review protocol
Systematic reviews should work to a pre-defined plan (or 'protocol') in order to reduce the risk of bias in the process. The protocol guides your process and prevents you from answering a different question to the one you set out to answer.
Writing a protocol
A protocol should describe:
- the question being asked
- the resources and approaches to identifying studies that the authors intend to use
- the inclusion/exclusion criteria against which the studies will be assessed
- the assessment criteria/tools to be applied to gauge individual study quality
- how the data from each study will be extracted and synthesised.
The PRISMA-P checklist is a useful guide to follow when writing a protocol.
-
PRISMA for systematic review protocols (PRISMA-P)PRISMA-P was published in 2015 aiming to facilitate the development and reporting of systematic review protocols. This site includes PRISMA-P checklist and an elaborating document.
Registering your protocol
Registering your protocol in a publicly accessible way will avoid other people duplicating your review. Similarly, it is always a good idea to check these sources ahead of starting out just in case someone else has lodged a review protocol on the same topic.
The main sources of registered protocols are listed here:
-
PROSPERO: International Prospective Register of Systematic ReviewsPROSPERO is an international database of prospectively registered systematic reviews in health and social care. Key features from the review protocol are recorded and maintained as a permanent record. PROSPERO aims to provide a comprehensive listing of systematic reviews registered at inception to help avoid unplanned duplication and enable comparison of reported review methods with what was planned in the protocol.
-
Joanna Briggs Institute - JBIAllows you to search a wide range of summarized and appraised evidence in health care and specialised health topics. This comprehensive range of resources is across seven publication types: Evidence Based Recommended Practices, Evidence Summaries, Best Practice Information Sheets, Systematic Reviews, Consumer Information Sheets, Systematic Review Protocols, and Technical Reports.
Quantitative Appraisal Tools
-
CASP UK: The Critical Appraisal Skills ProgrammeProvides learning resources and critical appraisal tools to use when reading research. CASP critical appraisal checklists are free to download and use under the Creative Commons License.
-
CEBM: Centre for Evidence-Based Medicine (Oxford, UK)A comprehensive online EBM resource. Includes critical appraisal checklists for a range of study designs.
-
JBI Critical Appraisal ToolsChecklists provided for a range of study types and designs. In addition to SRs, RCTs, and the major observational designs, JBI provides checklists for case reports, case series, prevalence studies, text and opinion works, and analytic cross-sectional studies.
-
Jadad scale for reporting randomized controlled trialsA procedure to assess the methodological quality of a randomised controlled trial. Published as: Jadad, Alejandro R. et al. 1996. Assessing the quality of reports of randomized clinical trials: Is blinding necessary? Controlled Clinical Trials, 17(1), 1-12.
-
Newcastle-Ottawa Scale (NOS)A tool for assessing the quality of non-randomised studies, including case control and cohort studies.
-
Cochrane Collaboration's tool for assessing risk of biasRead about the development and purpose of this tool in Higgins, JPT et al. 2011. The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials. BMJ, [Online]. 343, d5928. Available at: http://www.bmj.com/content/343/bmj.d5928 [Accessed 20 February 2015].
-
Evidence Analysis Manual (Academy of Nutrition and Dietetics)Provides quality criteria checklists for appraising both primary research and review articles.
-
REPOSE guidelines (EPPI-Centre)The EPPI-Centre's draft guidelines for the reporting of primary empirical research studies in education.
-
Quality Assessment tool for Quantitative StudiesThis EPHPP “Quality Assessment Tool for Quantitative Studies” method and tool was developed for use in public health, and can be applied to articles of any public health topic area, including the promotion of family and sexual health and the prevention of chronic disease, injuries and substance misuse. Various types of public health professionals would find this tool relevant to utilize sources of high quality literature to support the decision-making process, especially when designing, implementing and evaluating public health programs and policy.
Qualitative Appraisal Tools
Mixed Methods Appraisal Tools
Search
Search all our guides.