|Examples of Quality Checks||Examples of Bias|
(CASP) Critical Appraisal Skills Programme; Effective Practice and Organisation of Care Reviews (EPOC); Cochrane risk of Bias; Centre for Evidence Based Medicine critical Appraisal tools (CEBM); SURE for Randomized Controlled Trials and Other Experimental Studies; (SIGN) Scottish Intercollegiate Guidelines Network; (NHLBI) National Heart, Lung and Blood Institute; Joanna Briggs Checklist for Randomized Controlled Trials; PEDro Scale.
(CASP) Critical Appraisal Skills Programme; Effective Practice and Organisation of Care Reviews (EPOC); Cochrane ROBINS; (SURE) Specialist Unit for Review Evidence all quality appraisal tools; Joanna Briggs Checklists; (SIGN) Scottish Intercollegiate Guidelines Network; (NHLBI) National Heart, Lung and Blood Institute
(CASP) Critical Appraisal Skills Programme;; (SURE) Specialist Unit for Review Evidence all quality appraisal tools;; Joanna Briggs Checklists; Centre for Evidence Based Medicine critical Appraisal tools (CEBM)
Example of a more detailed Data Extraction Template from Cochrane here
In most systematic reviews of quantitative studies data extraction is a relatively linear process. Key items for data extraction are specified in advance in a data extraction template, based on the participants, interventions, comparisons and outcomes of interest. This template is then applied to each included study.
Programs like Excel or Google Spreadsheets may be the best option for smaller while specialist systematic review tools provide more detailed and comprehensive support.
It is recommended that to pilot your data extraction tool, especially if you will code your data, to determine if fields should be added or clarified, or if the review team needs guidance in collecting and coding data.
Extraction Templates and Tools
The Systematic Review Toolbox is a web-based catalogue of tools that support various tasks within the systematic review and wider evidence synthesis process. It was orignally developed by Dr Christopher Marshall and Anthea Sutton, launching in 2014.
Excel is the most basic tool for the management of the screening and data extraction stages of the systematic review process. Customized workbooks and spreadsheets can be designed for the review process..
The Systematic Review Data Repository (SRDR) is a powerful and easy-to-use tool for the extraction and management of data for systematic review or meta-analysis. It is also an open and searchable archive of systematic reviews and their data.
Covidence is a software platform built specifically for managing each step of a systematic review project, including data extraction.
RevMan is free software used to manage Cochrane reviews..
Distiller (needs subscription)
EPPI Reviewer (subscription, free trial)
Example of a Cochrane Data Extraction Template
Data Extraction Tips: Blog post from UK research group on issues to consider regarding data extraction.
Information to be extracted may include:
Bibliographic information (author, year);
Study type: systematic reviews of randomized controlled trials; randomized controlled trials; systematic reviews of non-randomized or cohort studies; cohort studies; case control studies; case series; case report
Participant information: (age, gender, sample size, whether from a high risk group, etc)
Intervention information (number of techniques used, use of multiple sessions, duration of intervention (weeks), format of delivery (individual, group, or mixed), source of delivery (medically trained health professional, non-medically trained health professional or non-health professional), theoretical background);
Methodological information (attrition, outcomes, how outcome was validated, length of follow up, study design);
Effect size information (mean, standard deviation, statistic type, value of statistic, p-value, direction of effect, number of responders).
Other possible categories:
Location setting (school, family, community wide, environmental), country);
Behavioural targets of intervention (physical activity, health eating)
Economic information: costs of the intervention, existing evaluation of cost-effectiveness, cost-benefit;
NB: Differentiate between dichotomous and continuous outcomes
Example of an outcomes data extraction table in a systematic review.
An specific analytical method to be used should be proposed at the protocol stage
Main analytical methods are: Meta-analysis; Narrative synthesis; Mixed methods; and Qualitative evidence synthesis.
refers to an approach to the systematic review and synthesis of findings from multiple studies that relies primarily on the use of words and text to summarise and explain the findings of the synthesis
Format/ Structure example:
|Mixed Methods||"The mixed methods approach to conducting systematic reviews is a process whereby (1) comprehensive syntheses of two or more types of data (e.g. quantitative and qualitative) are conducted and then aggregated into a final, combined synthesis, or (2) qualitative and quantitative data are combined and synthesized in a single primary synthesis." (The Joanna Briggs Institute 2014 Reviewers Manual)|
|Meta-analysis||A meta-analysis is the use of statistical methods to summarize the results of the studies selected as significant in a Systematic Review|
A qualitative evidence synthesis (commonly referred to as QES) can add value by providing decision makers with additional evidence to improve understanding of intervention complexity, contextual variations, implementation, and stakeholder preferences and experiences....
One type of Qualitative synthesis is Mata-ethnography, see: Meta-ethnography in healthcare research: a guide to using a meta-ethnographic approach for literature synthesis
Methods for qualitative evidence synthesis are complex and continue to develop. Authors should always consult current methods guidance at methods.cochrane.org/qi.
A meta-analysis is a statistical combination of quantitative results from similar studies.
[For a detailed overview see: Cochrane Handbook: Chapter 10 ]
Outcome types for analysis:
Dichotomous: Yes/No; alive/dead; success/failure:
Continuous: BMI; blood pressure; psychological scale etc
In the case of dichotomous outcomes the four pieces of data required for a meta-analysis are:
Control total number (data input 1) and event number (data input 2) and Experimental total number (data input 3) and event number (data input 4).
In the case of continuous outcomes an (SD) standard deviation is required, this can sometimes be inferred from other data.
Other statistical relevant calculations can be inferred from the control (total and event numbers) and Experimental (control and event numbers) and include:
|Odds||Odds(event) = Prob(event)/Prob(Not event)|
|Odds(event) Control group /Odds(event)Treatment group|
|Relative Risk||Prob(event) Control group / Prob(event) Treatment group|
Example of a date outcome statistics figure in a systematic review from which 'Experimental ' and 'Control' total and standard deviation numbers can be extrapolated
Statistical packages such as RevMan (free from Cochrane) can be used to preform the statistical analysis and produce a forest plot.
RevMan5 showing statistical analysis options and input tables to allow production of a forest plot.
Note: As is shown in the 'Statistical Input in RevMan diagram only 4 pieces of data input are required (input boxes outlined in green), to produce a forest plot!
|Statistical Options in RevMan||Statistical Input in RevMan|
A forest plot is:the graphical display of individual study results and, the weighted average of studies included in a systematic review. A forest plot is one way of summarizing the review's results for a specific outcome.
Forest plot example:
Example of a forest plot figure in a systematic review from Cochrance: (Extension of figure four from the studyshown already)