Definition
Meta-analysis is a statistical procedure used to combine data from different studies with same research question to come to a more accurate conclusion. It usually includes the randomized controlled trials (RCTs) but sometimes may include cross-sectional and epidemiological studies.
Different studies with same research question have variable results, especially if the sample size is small. In such scenarios, the chances of error increase. Meta-analysis objectively interprets data from all the relevant studies to give us a precise answer to the question under research.
How Different Is Meta-Analysis from Systemic Review?
A systemic review is an objective and reproducible method to review all the relevant evidence on the literature available as per the eligibility criteria that is predefined.
Meta-analysis is a statistical method used to combine numerical results from all the studies and it is usually a part of systemic analysis.
In the hierarchy of evidence, systemic review and meta-analysis sit at top and provide enough evidence for medical practice, thereby giving way for evidence-based medicine. Evidence-based medicine can be defined as systemic, experimental approach to use the current evidence for obtaining and using medical information, and using it in making decision for the treatment of each individual patient.
Meta-analyses were rare in literature before 70s but since then there has been an exponential growth. Since 1999, various papers have presented guidelines for reporting meta-analyses of RCTs. QUORUM (Quality of Reporting of Meta-analyses) statement gave the initial guidelines which is replaced by PRISMA (Preferred Reporting Items for Systemic Review and Meta-Analyses) statement in 2009.
Following are the steps of a meta-analysis:
-
Formulating a research hypothesis/question: Multiple studies may have different results regarding the same study question that makes them impossible to compare and use their results for the benefit of patients. So, combining these similar studies to give the combined analysis of them all provides the results that can be helpful for medical practice.
-
Protocol and registration: Registration of each study is very important to make the research transparent. It also includes noting primary and secondary outcomes; and methods along with any deviations from protocol which are informed to the researchers and readers with reasons specified. Primary and secondary outcomes and methods and if any deviation from protocol are informed to the researchers and readers with reasons specified.
-
Defining inclusion and exclusion criteria: Studies are selected on the basis of inclusion and exclusion criteria. Two independent assessors decide independently which studies to be included and excluded. When a study is excluded from meta-analyses, the reason is provided for the exclusion. This is followed by consensus meeting to discuss the studies excluded or included.
Every study is evaluated for quality and protocol design. To minimize the bias, it is important to include the negative trials or the unpublished trials too.
-
Literature search and study selection: A good meta-analyses requires a thorough literature search. As per the PRISMA statement, each study included should have information about patient characteristics, interventions, comparisons, outcome, and study design (PICOS). It is important to include all the positive or negative, published or unpublished studies to minimize the bias. PRISMA statement recommends search for one single database in total among many, for example, PUBMED, ScienceDirect, CENTRAL (Cochrane Central Register for RCT), Google Scholar, and ISI web of knowledge.[1] It is not easy to obtain data on unpublished studies, which can be obtained from clinical trial registers.
Data obtained from these is of two types: aggregated and individual. Individual data from RCT is the gold standard to be utilized in meta-analyses and is the best way to obtain a more global picture of the natural history and predictors of risk for major outcomes. It has greater validity compared with aggregated data. It is easier to conduct appropriate statistical analysis and to make comparison among different subgroups ([Fig. 1]).[2]
-
Quality of evidence: The quality of evidence is evaluated on the basis of the study limitations, inaccuracies, incompleteness of outcome data, indirectness of evidence, and risk of publication bias that is obtained using GRADE (Grading of Recommendations, Assessment, Development and Evaluations) system.[3]
-
Data extraction: The data extracted is reviewed from each study and certain changes are made to combine the data as the size and format of each variable are different. If it is not possible to combine, then the analysis is limited to systemic review. These difficulties in combining data are by the investigators by debate and if not resolved, a third investigator is consulted.
-
Analyzing data: Depending upon the type of data, the analysis takes place. The most common measures of effect used for dichotomous data are the risk ratio (also called relative risk) and the odds ratio. It is helpful in calculation of event occurred in control and intervention group and the difference gives absolute relative risk given by 2x2 Contingency table.
Fig. 1 Preferred Reporting Items for Systemic Review and Meta-Analyses (PRISMA) 2009 for selection of studies in four phases.[2]
Analysis of effect in a meta-analysis is divided on the basis of the fixed effect model or random effect model.[3]
Fixed effect model: The difference in the effect of treatment is minor or same and is due to random error only, where most studies have similar characteristics of patients and methodology.
Random effect model: It assumes that the size of the effect of treatment differs among studies and the differences in variation among studies are not only due to random error but also due to variability of the patients included and the methodology or the statistical methods used to reach the final result. It has studies with great heterogenicity.
Heterogenicity
[3]: It is the variability among the studies with the similar study question. If heterogenicity is present, one must think how to generalize the results of meta-analysis. The analysis of factors leading to heterogenicity in each study can lead to new study question. So, it can be defined statistically as the variation in the effect that is more than that given only by the random error.
Heterogenicity can be due to formulation of a research hypothesis/question. Clinical variation—patient characteristics or intervention used and protocol and registration and statistical variation—method of analysis, methodology of the study, study design, and blinding can lead to different outcome of effect.
Higher the heterogenicity, difficult it is to generalize the studies for meta-analysis, but it can help in analysis of data for newer study questions. So, meta-analysis should be conducted when a group of studies is sufficiently homogeneous in terms of subjects involved, interventions, and outcomes to provide a meaningful summary. It is difficult to generalize the results of the analysis if the studies included are diverse and have high heterogenicity.
Publication Bias: Publication bias is also caused due to the nonpublication of negative studies. Journals tend to not accept negative trials and this is also an important factor for publication bias. It is the most common bias in meta-analysis. It occurs due to most of the studies published with statistically significant effect and the nonsignificant ones not being published.
The larger studies will have lower standard error and tend to cluster, closer to the point estimate, whereas the small studies will be lesser precise and will be widely distributed with highly variable results. Funnel plots are used to evaluate for publication bias in a meta-analysis.
To negate publication bias, funnel plot is useful. Symmetrical funnel plots mean both positive and negative studies are equally represented in meta-analysis. Asymmetrical funnel plots are due to publication bias and may also be due to heterogenicity of the studies that represent that nonsignificant studies have been included ([Fig. 2]).
Fig. 2 (A) Symmetrical funnel plot—wide base and narrow top. (B) Asymmetrical funnel plot—studies distributed to one side.
Other biases noted are lag time bias, language bias, and selective reporting bias.
-
Result presentation: The results of a meta-analysis are presented in form of FOREST PLOT. Forest plot can not only give the overall result of the analysis but also the cumulative meta-analysis result.
Also, the p-values for the null hypothesis by the Z test and p-value for heterogenicity by the chi-squared test are provided. A p-value by the z test <0.5 means significant difference in the effect of two treatment methods and >0.5 means no significant difference in the effect exists.
|
Advantages
|
Disadvantages
|
|
Cochrane reviews are one of the best examples of systemic reviews in health care sector. They have a fixed set of rules of doing systemic reviews that allow only high-quality studies and minimize bias across all parts of review process.
|
|
1. Evaluation of large number of studies having the same study question
2. Increases the strength of conclusions with good trial designs
3. Results can be generalized to general population with clinical benefit
4. Improves precision and accuracy by involving more datasets
5. Focusing on the trial helps in providing reasoning for the inconsistent results
6. Increases the statistical power to detect an effect
7. Variations can be explained by the assessors
8. Presence of publication bias can be known
|
1. Time consuming
2. Only studies that are published or abstracts available are included
3. Inclusion of small studies may not give conclusive results
4. Cannot control the source of bias
5. Can have publication, language, selective reporting bias
6. Larger heterogenicity in studies included will not lead to conclusive results
|
For all medical oncologists, EBCTCG meta-analysis is very well known. It gives answers to most questions pertaining to early breast cancer. They invite data from almost all RCTs across globe, have large-scale randomization, and have long follow-ups. They have huge datasets and thus are most reliable source in giving estimates of differences in various treatment approaches.
Network meta-analysis is a type of meta-analysis in which various treatments (more than three) are compared. Comparison is made directly within RCT or indirectly across trials having common comparator arm. To have valid findings, it is to be designed and conducted rigorously. It is especially useful in situations where there is paucity of many RCTs comparing treatments.
Meta-analysis and systemic reviews are one of the best analytic methods available in modern evidence-based medicine era providing the highest evidence for the study question.
