Systematic Review | Definition, Examples & Guide
A systematic review is a type of review that uses repeatable methods to find, select, and synthesise all available evidence. It answers a clearly formulated research question and explicitly states the methods used to arrive at the answer.
They answered the question ‘What is the effectiveness of probiotics in reducing eczema symptoms and improving quality of life in patients with eczema?’
In this context, a probiotic is a health product that contains live microorganisms and is taken by mouth. Eczema is a common skin condition that causes red, itchy skin.
They used systematic methods to find, select, and synthesise all available evidence, and they described these methods in detail in their article. Based on the evidence, Boyle and colleagues concluded that probiotics cannot be recommended for reducing eczema symptoms or improving quality of life in patients with eczema.
Table of contents
What is a systematic review?
A review is an overview of the research that’s already been completed on a topic.
What makes a systematic review different from other types of reviews is that the research methods are designed to reduce research bias. The methods are repeatable, and the approach is formal and systematic:
- Formulate a research question
- Develop a protocol
- Search for all relevant studies
- Apply the selection criteria
- Extract the data
- Synthesise the data
- Write and publish a report
Although multiple sets of guidelines exist, the Cochrane Handbook for Systematic Reviews is among the most widely used. It provides detailed guidelines on how to complete each step of the systematic review process.
Systematic reviews are most commonly used in medical and public health research, but they can also be found in other disciplines.
Systematic reviews typically answer their research question by synthesising all available evidence and evaluating the quality of the evidence. Synthesising means bringing together different information to tell a single, cohesive story. The synthesis can be narrative (qualitative), quantitative, or both.
Systematic review vs meta-analysis
Systematic reviews often quantitatively synthesise the evidence using a meta-analysis. A meta-analysis is a statistical analysis, not a type of review.
A meta-analysis is a technique to synthesise results from multiple studies. It’s a statistical analysis that combines the results of two or more studies, usually to estimate an effect size.
Systematic review vs literature review
A literature review is a type of review that uses a less systematic and formal approach than a systematic review. Typically, an expert in a topic will qualitatively summarise and evaluate previous work, without using a formal, explicit method.
Although literature reviews are often less time-consuming and can be insightful or helpful, they have a higher risk of bias and are less transparent than systematic reviews.
Systematic review vs scoping review
Similar to a systematic review, a scoping review is a type of review that tries to minimise bias by using transparent and repeatable methods.
However, a scoping review isn’t a type of systematic review. The most important difference is the goal: rather than answering a specific question, a scoping review explores a topic. The researcher tries to identify the main concepts, theories, and evidence, as well as gaps in the current research.
Sometimes scoping reviews are an exploratory preparation step for a systematic review, and sometimes they are a standalone project.
When to conduct a systematic review
A systematic review is a good choice of review if you want to answer a question about the effectiveness of an intervention, such as a medical treatment.
To conduct a systematic review, you’ll need the following:
- A precise question, usually about the effectiveness of an intervention. The question needs to be about a topic that’s previously been studied by multiple researchers. If there’s no previous research, there’s nothing to review.
- A team of at least three people. Best practices require three people for certain steps of the systematic review process. Ideally, in addition to your research team you’ll also have an advisory group of about six people.
- If you’re doing a systematic review on your own (e.g., for a research paper or thesis), you should take appropriate measures to ensure the validity and reliability of your research.
- Access to databases and journal archives. Often, your educational institution provides you with access.
- Time. A professional systematic review is a time-consuming process: it will take the lead author about six months of full-time work. If you’re a student, you should narrow the scope of your systematic review and stick to a tight schedule.
- Bibliographic, word-processing, spreadsheet, and statistical software. For example, you could use EndNote, Microsoft Word, Excel, and SPSS.
Pros and cons of systematic reviews
A systematic review has many pros.
- They minimise research bias by considering all available evidence and evaluating each study for bias.
- Their methods are transparent, so they can be scrutinised by others.
- They’re thorough: they summarise all available evidence.
- They can be replicated and updated by others.
Systematic reviews also have a few cons.
- They’re time-consuming.
- They’re narrow in scope: they only answer the precise research question.
Step-by-step example of a systematic review
The 7 steps for conducting a systematic review are explained with an example.
Step 1: Formulate a research question
Formulating the research question is probably the most important step of a systematic review. A clear research question will:
- Allow you to more effectively communicate your research to other researchers and practitioners
- Guide your decisions as you plan and conduct your systematic review
A good research question for a systematic review has four components, which you can remember with the acronym PICO:
- Population(s) or problem(s)
- Intervention(s)
- Comparison(s)
- Outcome(s)
You can rearrange these four components to write your research question:
- What is the effectiveness of I versus C for O in P?
Sometimes, you may want to include a fourth component, the type of study design. In this case, the acronym is PICOT.
- Type of study design(s)
- The population of patients with eczema
- The intervention of probiotics
- In comparison to no treatment, placebo, or non-probiotic treatment
- The outcome of changes in participant-, parent-, and doctor-rated symptoms of eczema and quality of life
- Randomised control trials, a type of study design
Their research question was:
- What is the effectiveness of probiotics versus no treatment, a placebo, or a non-probiotic treatment for reducing eczema symptoms and improving quality of life in patients with eczema?
Step 2: Develop a protocol
A protocol is a document that contains your research plan for the systematic review. This is an important step because having a plan allows you to work more efficiently and reduces bias.
Your protocol should include the following components:
- Background information: Provide the context of the research question, including why it’s important.
- Research objective(s): Rephrase your research question as an objective.
- Proposed methods
- Selection criteria: State how you’ll decide which studies to include or exclude from your review.
- Search strategy: Discuss your plan for finding studies.
- Analysis: Explain what information you’ll collect from the studies and how you’ll synthesise the data.
If you’re a professional seeking to publish your review, it’s a good idea to bring together an advisory committee. This is a group of about six people who have experience in the topic you’re researching. They can help you make decisions about your protocol.
It’s highly recommended to register your protocol. Registering your protocol means submitting it to a database such as PROSPERO or ClinicalTrials.gov.
Step 3: Search for all relevant studies
Searching for relevant studies is the most time-consuming step of a systematic review.
To reduce bias, it’s important to search for relevant studies very thoroughly. Your strategy will depend on your field and your research question, but sources generally fall into these four categories:
- Databases: Search multiple databases of peer-reviewed literature, such as PubMed or Scopus. Think carefully about how to phrase your search terms and include multiple synonyms of each word. Use Boolean operators if relevant.
- Handsearching: In addition to searching the primary sources using databases, you’ll also need to search manually. One strategy is to scan relevant journals or conference proceedings. Another strategy is to scan the reference lists of relevant studies.
- Grey literature: Grey literature includes documents produced by governments, universities, and other institutions that aren’t published by traditional publishers. Graduate student theses are an important type of grey literature, which you can search using the Networked Digital Library of Theses and Dissertations (NDLTD). In medicine, clinical trial registries are another important type of grey literature.
- Experts: Contact experts in the field to ask if they have unpublished studies that should be included in your review.
At this stage of your review, you won’t read the articles yet. Simply save any potentially relevant citations using bibliographic software, such as Scribbr’s APA or MLA Generator.
- Databases: EMBASE, PsycINFO, AMED, LILACS, and ISI Web of Science
- Handsearch: Conference proceedings and reference lists of articles
- Grey literature: The Cochrane Library, the metaRegister of Controlled Trials, and the Ongoing Skin Trials Register
- Experts: Authors of unpublished registered trials, pharmaceutical companies, and manufacturers of probiotics
Step 4: Apply the selection criteria
Applying the selection criteria is a three-person job. Two of you will independently read the studies and decide which to include in your review based on the selection criteria you established in your protocol. The third person’s job is to break any ties.
To increase inter-rater reliability, ensure that everyone thoroughly understands the selection criteria before you begin.
If you’re writing a systematic review as a student for an assignment, you might not have a team. In this case, you’ll have to apply the selection criteria on your own; you can mention this as a limitation in your paper’s discussion.
You should apply the selection criteria in two phases:
- Based on the titles and abstracts: Decide whether each article potentially meets the selection criteria based on the information provided in the abstracts.
- Based on the full texts: Download the articles that weren’t excluded during the first phase. If an article isn’t available online or through your library, you may need to contact the authors to ask for a copy. Read the articles and decide which articles meet the selection criteria.
It’s very important to keep a meticulous record of why you included or excluded each article. When the selection process is complete, you can summarise what you did using a PRISMA flow diagram.
Next, Boyle and colleagues found the full texts for each of the remaining studies. Boyle and Tang read through the articles to decide if any more studies needed to be excluded based on the selection criteria.
When Boyle and Tang disagreed about whether a study should be excluded, they discussed it with Varigos until the three researchers came to an agreement.
After applying the selection criteria, they were left with 12 studies involving 781 participants.
Step 5: Extract the data
Extracting the data means collecting information from the selected studies in a systematic way. There are two types of information you need to collect from each study:
- Information about the study’s methods and results. The exact information will depend on your research question, but it might include the year, study design, sample size, context, research findings, and conclusions. If any data are missing, you’ll need to contact the study’s authors.
- Your judgement of the quality of the evidence, including risk of bias.
You should collect this information using forms. You can find sample forms in The Registry of Methods and Tools for Evidence-Informed Decision Making and the Grading of Recommendations, Assessment, Development and Evaluations Working Group.
Extracting the data is also a three-person job. Two people should do this step independently, and the third person will resolve any disagreements.
They also collected data about possible sources of bias, such as how the study participants were randomised into the control and treatment groups.
If Boyle and Tang disagreed on anything, then Varigos broke the tie. Boyle and Bath-Hextall entered the data from the forms into a computer spreadsheet.
Step 6: Synthesise the data
Synthesising the data means bringing together the information you collected into a single, cohesive story. There are two main approaches to synthesising the data:
- Narrative (qualitative): Summarise the information in words. You’ll need to discuss the studies and assess their overall quality.
- Quantitative: Use statistical methods to summarise and compare data from different studies. The most common quantitative approach is a meta-analysis, which allows you to combine results from multiple studies into a summary result.
Generally, you should use both approaches together whenever possible. If you don’t have enough data, or the data from different studies aren’t comparable, then you can take just a narrative approach. However, you should justify why a quantitative approach wasn’t possible.
Boyle and colleagues also divided the studies into subgroups, such as studies about babies, children, and adults, and analysed the effect sizes within each group.
Boyle and colleagues’ meta-analysis found that there was no significant effect of probiotic treatment on eczema symptoms, either overall or in any subgroup.
Step 7: Write and publish a report
The purpose of writing a systematic review article is to share the answer to your research question and explain how you arrived at this answer.
Your article should include the following sections:
- Abstract: A summary of the review
- Introduction: Including the rationale and objectives
- Methods: Including the selection criteria, search method, data extraction method, and synthesis method
- Results: Including results of the search and selection process, study characteristics, risk of bias in the studies, and synthesis results
- Discussion: Including interpretation of the results and limitations of the review
- Conclusion: The answer to your research question and implications for practice, policy, or research
To verify that your report includes everything it needs, you can use the PRISMA checklist.
Once your report is written, you can publish it in a systematic review database, such as the Cochrane Database of Systematic Reviews, and/or in a peer-reviewed journal.
In their report, Boyle and colleagues concluded that probiotics cannot be recommended for reducing eczema symptoms or improving quality of life in patients with eczema.
Frequently asked questions about systematic reviews
- Is a systematic review primary research?
-
A systematic review is secondary research because it uses existing research. You don’t collect new data yourself.
- What is a literature review?
-
A literature review is a survey of scholarly sources (such as books, journal articles, and theses) related to a specific topic or research question.
It is often written as part of a dissertation, thesis, research paper, or proposal.
- What is the purpose of a literature review?
-
There are several reasons to conduct a literature review at the beginning of a research project:
- To familiarise yourself with the current state of knowledge on your topic
- To ensure that you’re not just repeating what others have already done
- To identify gaps in knowledge and unresolved problems that your research can address
- To develop your theoretical framework and methodology
- To provide an overview of the key findings and debates on the topic
Writing the literature review shows your reader how your work relates to existing research and what new insights it will contribute.
Cite this Scribbr article
If you want to cite this source, you can copy and paste the citation or click the ‘Cite this Scribbr article’ button to automatically add the citation to our free Reference Generator.
Turney, S. (2024, July 17). Systematic Review | Definition, Examples & Guide. Scribbr. Retrieved 9 December 2024, from https://www.scribbr.co.uk/research-methods/systematic-reviews/
