Background:
There is a discrepancy between the outcome of a meta-analysis published in 1997 of
89 trials of homeopathy by Linde et al and an analysis of 110 trials by Shang et al
published in 2005, these reached opposite conclusions. Important data were not mentioned
in Shang et al's paper, but only provided subsequently.
Questions:
What was the outcome of Shang et al's predefined hypotheses? Were the homeopathic
and conventional trials comparable? Was subgroup selection justified? The possible
role of ineffective treatments. Was the conclusion about effect justified? Were essential
data missing in the original article?
Methods:
Analysis of post-publication data. Re-extraction and analysis of 21 higher quality
trials selected by Shang et al with sensitivity analysis for the influence of single
indications. Analysis of comparability. Sensitivity analysis of influence of subjective
choices, like quality of single indications and of cut-off values for ‘larger samples’.
Results:
The quality of trials of homeopathy was better than of conventional trials. Regarding
smaller trials, homeopathy accounted for 14 out of 83 and conventional medicine 2 out
of 78 good quality trials with n < 100. There was selective inclusion of unpublished trials only for homeopathy. Quality
was assessed differently from previous analyses. Selecting subgroups on sample size
and quality caused incomplete matching of homeopathy and conventional trials. Cut-off
values for larger trials differed between homeopathy and conventional medicine without
plausible reason. Sensitivity analyses for the influence of heterogeneity and the
cut-off value for ‘larger higher quality studies’ were missing. Homeopathy is not
effective for muscle soreness after long distance running, OR = 1.30 (95% CI 0.96–1.76).
The subset of homeopathy trials on which the conclusion was based was heterogeneous,
comprising 8 trials on 8 different indications, and was not matched on indication
with those of conventional medicine. Essential data were missing in the original paper.
Conclusion:
Re-analysis of Shang's post-publication data did not support the conclusion that
homeopathy is a placebo effect. The conclusion that homeopathy is and that conventional
is not a placebo effect was not based on comparative analysis and not justified because
of heterogeneity and lack of sensitivity analysis. If we confine ourselves to the
predefined hypotheses and the part of the analysis that is indeed comparative, the
conclusion should be that quality of homeopathic trials is better than of conventional
trials, for all trials (p = 0.03) as well as for smaller trials (p = 0.003).
Keywords
homeopathy - meta-analysis - comparative analysis - quality bias - selection bias
- cut-off value - adverse effects