The 2005 meta-analysis of homeopathy: the importance of post-publication data

Background: There is a discrepancy between the outcome of a meta-analysis published in 1997 of 89 trials of homeopathy by Linde et al and an analysis of 110 trials by Shang et al published in 2005, these reached opposite conclusions. Important data were not mentioned in Shang et al's paper, but only provided subsequently.

Questions: What was the outcome of Shang et al's predefined hypotheses? Were the homeopathic and conventional trials comparable? Was subgroup selection justified? The possible role of ineffective treatments. Was the conclusion about effect justified? Were essential data missing in the original article?

Methods: Analysis of post-publication data. Re-extraction and analysis of 21 higher quality trials selected by Shang et al with sensitivity analysis for the influence of single indications. Analysis of comparability. Sensitivity analysis of influence of subjective choices, like quality of single indications and of cut-off values for ‘larger samples’.

Results: The quality of trials of homeopathy was better than of conventional trials. Regarding smaller trials, homeopathy accounted for 14 out of 83 and conventional medicine 2 out of 78 good quality trials with n < 100. There was selective inclusion of unpublished trials only for homeopathy. Quality was assessed differently from previous analyses. Selecting subgroups on sample size and quality caused incomplete matching of homeopathy and conventional trials. Cut-off values for larger trials differed between homeopathy and conventional medicine without plausible reason. Sensitivity analyses for the influence of heterogeneity and the cut-off value for ‘larger higher quality studies’ were missing. Homeopathy is not effective for muscle soreness after long distance running, OR = 1.30 (95% CI 0.96–1.76). The subset of homeopathy trials on which the conclusion was based was heterogeneous, comprising 8 trials on 8 different indications, and was not matched on indication with those of conventional medicine. Essential data were missing in the original paper.

Conclusion: Re-analysis of Shang's post-publication data did not support the conclusion that homeopathy is a placebo effect. The conclusion that homeopathy is and that conventional is not a placebo effect was not based on comparative analysis and not justified because of heterogeneity and lack of sensitivity analysis. If we confine ourselves to the predefined hypotheses and the part of the analysis that is indeed comparative, the conclusion should be that quality of homeopathic trials is better than of conventional trials, for all trials (p = 0.03) as well as for smaller trials (p = 0.003).

Keywords

homeopathy - meta-analysis - comparative analysis - quality bias - selection bias - cut-off value - adverse effects

• References

• 1 Kleijnen J., Knipschild P., ter Riet G. Clinical trials of homeopathy. BMJ 1991; 302: 316-323.
• 2 Linde K., Clausius N., Ramirez G. et al. Are the clinical effects of homeopathy placebo effects? A meta-analysis of placebo-controlled trials. Lancet 1997; 350: 834-843.
• 3 Cucherat M., Haugh M., Gooch M., Boissel J. Evidence of clinical efficacy of homeopathy - A meta-analysis of clinical trials. Eur J Clin Pharmacol 2000; 56: 27-33.
• 4 Morrison B., Lilford R.J., Ernst E. Methodological rigour and results of clinical trials of homeopathic medicines. Perfusion 2000; 13: 132-138.
• 5 Ernst E. A systematic review of systematic reviews of homeopathy. J Clin Pharmacol 2002; 54: 577-582.
• 6 Ezzo J., Bausell B., Moerman D.E., Berman B., Hadhazy V. Reviewing the reviews. How strong is the evidence? How clear are the conclusions?. Int J Technol Assess Health Care 2001; Fall; 17 (04) 457-466.
• 7 Vandenbroucke J.P. Medical journals and the shaping of medical knowledge. Lancet 1998; 352: 2001-2006.
• 8 Sterne J.A.C., Egger M., Smith G.D. Investigating and dealing with publication and other biases in meta-analysis. BMJ 2001; 323: 101-105.
• 9 Shang A., Huwiler-Müntener K., Nartey L. et al. Are the clinical effects of homeopathy placebo effects? Comparative study of placebo-controlled trials of homeopathy and allopathy. Lancet 2005; 366: 726-732.
• 10 Vandenbroucke J.P. Homeopathy and the growth of truth. Lancet 2005; 366: 691-692.
• 11 In: Higgins J.P.T., Green S. Cochrane Handbook for Systematic Reviews of Interventions 4.2.5. The Cochrane Library 2005 (Issue 3). Chichester, UK: John Wiley & Sons, Ltd [updated May 2005].
• 12 Jonas W.B., Kaptchuk T.J., Linde K. A critical overview of homeopathy. Ann Intern Med 2003; 138: 393-399.
• 13 Egger M., Smith G.D., Altman D.G. Systematic reviews in health care: meta-analysis in context. 2nd edition. BMJ Books; 2001.
• 14 Boden W.E. Meta-analysis in clinical trials reporting: has a tool become a weapon?. Am J Cardiol 1992; 69: 681-686.
• 15 Walach H., Jonas W., Lewith G. Letter to the editor. Lancet 2005; 366: 2081.
• 16 Linde K., Jonas W.B. Letter to the editor. Lancet 2005; 366: 2081-2082.
• 17 Fisher P., Berman B., Davidson J., Reilly D., Thompson T. et al. Letter to the editor. Lancet 2005; 366: 2082-2083.
• 18 Lüdtke R., Rutten A.L.B. The conclusions on the effectiveness of homeopathy highly depend on the set of analysed trials. J Clin Epid 2008 10.1016/j.jclinepi.2008.06.015.
• 19 Gade J., Thorn P. Paraghurt for patients with irritable bowel syndrome: a controlled clinical investigation from general practice. Scand J Prim Health Care 1989; 7: 23-26.
• 20 Humphrey R.G., Bartfield M.C., Carlan S.J., O'Brien W.F., O'Leary T.D., Triana T. Sulindac to prevent recurrent preterm labor: a randomized controlled trial. Obstet Gynecol 2001; 98: 555-562.
• 21 Chan A.W., Hjobartsson A., Haahr M.T., Gotzsche P.C., Altman D.G. Emperical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles. JAMA 2004; 291: 2457-2465.
• 22 Wiesenauer M., Gaus W. Wirksamkeitsnachweis eines Homöopathikums bei chronischer Polyarthritis. Eine randomisierte Doppelblindstudie bei niedergelassenen Ärzten. Aktuelle Rheumatologie 1991; 16: 1-9.
• 23 Schmidt J.M., Ostermayr B. Does a homeopathic ultramolecular dilution of Thyroidinum 30CH affect the decrease of body weight reduction in fasting patients? - A randomised Placebo-controlled double-blind trial. Homeopathy 2002; 91 (04) 197-206.
• 24 Enzi G., Crepaldi G., Inelmen E.M., Bruni R., Baggio B. Efficacy and safety of dexfenfluramine in obese patients: amulticenter study. Clin Neuropharmacology 1995; 12: S173-S178.
• 25 Available from: <http://www.fda.gov/CDER//news/phen/fenphenpr81597.htm>.
• 26 Louviere R.L., Upton R.T. Evaluation of Deladumone OB in the suppression of postpartum lactation. Am J Obstet Gynecol 1975; 121: 641-642.
• 27 Lacey P.H., Dodd G.D., Shannon D.J. A double blind, placebo controlled study of piroxicam in the management of acute muscoloskeletal disorders. Eur J Rheum Inflam 1984; 7: 95-104.
• 28 Available from: <http://www.fda.gov/ohrms/dockets/98fr/102998b.pdf>.
• 29 EMEA. Press release. London: European Medicines Agency Recommends Restricted Use for Piroxicam; 25 June 2007.
• 30 Grio R., Cellura A., Geranio R., Porpiglia M., Piacentino R. Efficacia clinica del tamoxifene nel trattamento della mastodynia premenstruale. Min Ginecol 1998; 50: 101-103.
• 31 de Lange-de Klerk E.S.M. Effects of homeopathic medicines on children with recurrent upper respiratory tract infections. BMJ 1994; 309: 1329-1332.
• 32 Reilly D.T., Taylor M.A., McSharry C., Aitchison T. Is homeopathy a placebo response? Controlled trial of homeopathic potency with pollen in hay fever as model. Lancet 1986: 881-886.
• 33 Hofmeyr G.J., Picconi V., Blauhof P. Postpartum homeopathic Arnica montana: a potency-finding pilot study. Br J Clin Pract 1990; 44: 619-621.
• 34 Reilly D., Taylor M.A., Beattie N.G. et al. Is evidence for homeopathy reproducible?. Lancet 1994; 344: 1601-1606.
• 35 Schulz K.F., Chalmers I., Hayes R.J., Altman D. Emperical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA 1995; 273: 408-412.
• 36 Rottey E.E.D., Verleye G.B., Liagre R.L.P. Het effect van een homeopathische bereiding van micro-organismen bij de preventie von griepsymtomen - Een gerandomiseerd dubbel-blind onderzoek in de huisartspraktijk. Tijdschr Integ Geneeskunde 1995; 11: 54-58.
• 37 Vickers A.J., Fisher P., Wyllie S.E., Rees R. Homeopathic Arnica 30X Is Ineffective for Muscle Soreness After Long-Distance Running - A randomized, double-blind, Placebo-controlled trial. Clin J Pain 1998; 14 (03) 227-231.
• 38 Papp R., Schuback G., Beck E. et al. Oscillococcinum in patients with influenza-like syndroms: A placebo controlled double-blind evaluation. Brit Homeopath J 1998; 87 (02) 69-76.
• 39 Labrecque M., Audet D., Latulippe L., Drouin J. Homeopathic treatment of plantar warts. Can Med Assoc J 1992; 146 (10) 1749-1753.
• 40 Jacobs J., Jiménez L.M., Malthouse S. et al. Homeopathic Treatment of Acute Childhood Diarrhea - Results from a Clinical Trial in Nepal. J Alternat Complement Med 2000; 6 (02) 131-139.
• 41 Weiser M., Clasen B.P.E. Randomisierte plazebokontrollierte Doppelblindstudie zur Untersuchung der klinischen Wirksamkeit der homöopathischen Euphorbium compositum-Nasentropfen S bei chronischer Sinusitis. Forsch Komplementarmed 1994; 1: 251-259.
• 42 Walach H., Haeusler W., Lowes T. et al. Classical homeopathic treatment of chronic headaches. Cephalalgia 1997; 17: 119-126.
• 43 Jacobs J., Jiménez L.M., Gloyd S.S., Gale J.L., Crothers D. Treatment of Acute Childhood Diarrhea With Homoeopathic Medicine: A Randomized Clinical Trial in Nicaragua. Pediatrics 1994; 93 (05) 719-725.
• 44 Jacobs J., Springer D.A., Crothers D. Homeopathic treatment of acute otitis media in children: a preliminary randomized placebo- controlled trial. Pediatr Infect Dis J 2001: 177-183.
• 45 Hart O., Mullee M.A., Lewith G., Miller J. Double-blind, placebo-controlled, randomized clinical trial of homoeopathic arnica C30 for pain and infection after total abdominal hysterectomy. J R Soc Med 1997; 90: 73-78.
• 46 Wiesenauer M., Häussler S., Gaus W. Pollinosis-Therapie mit Galphimia glauca. MMW Fortschr Med 1983; 101 (17) 811-814.
• 47 Zell J., Connert W.D., Mau J., Feuerstake G. Behandlung von akuten Sprunggelenksdistorsionen - Doppelblindstudie zum Wirksamkeitsnachweis eines homöopathischen Salbenpräparats. MMW Fortschr Med 1988; 106 (05) 96-100.
• 48 Böhmer D., Ambrus P. Behandlung von Sportverletzungen mit Traumeel-Salbe - Kontrollierte Doppelblindstudie. Biol Medizin 1992; 21 (04) 260-268.
• 49 Vickers A.J., Fisher P., Smith C., Wyllie S.E., Lewith G.T. Homoeopathy for delayed onset muscle soreness - A randomised double blind placebo controlled trial. Brit J Sports Med 1997; 31: 304-307.
• 50 Jawara N., Lewith G.T., Vickers Aj, Mullee M.A., Smith C. Homoeopathic Arnica and Rhus toxicodendron for delayed onset muscle soreness - A pilot for a randomized, double-blind, placebo-controlled trial. Brit Hom J 1997; 86 (01) 10-15.
• 51 Chapman E.H., Weintraub R.J., Milburn M.A. Homeopathic Treatment of Mild Traumatic Brain Injury - A Randomised, Double-Blind, Placebo- Controlled Clinical Trial. J Head Trauma Rehabil 1999; 14: 521-542.
• 52 Tveiten D., Bruset S., Borchgrevink C.F., Norseth J. Effects of the homeopathic remedy Arnica D30 on marathon runners: A randomized, double-blind study during the 1995 Oslo Marathon. Complement Ther Med 1998; 6 (02) 71-74.
• 53 Stevinson C., Devaraj V.S., Fountain-Barber A. homeopathic arnica for prevention of pain and bruising: randomized placebo-controlled trial in hand surgery. J R Soc Med 2003; 96: 60-65.
• 54 Lepaisant C. Essai thérapeutique en homéopathie: traitement des tensions mammaires et mastodynies du syndrome prémenstruel. Rev Fr Gynécol Obstét 1995; 90 (02) 94-97.
• 55 Chapman E.H., Angelica J., Spitalny G., Strauss M. Results of a study of the homoeopathic treatment of PMS. J Am Inst Hom 1994; 87: 14-21.
• 56 Kaplan M.A., Prior M.J., McKonly K.I., Du Pont H.L., Temple A.R., Nelson E.B. A multicenter randomized controlled trial of a liquid loperamide product versus placebo in the treatment of acute diarrhea in children. Clin Pediatr 1999; 38: 579-591.
• 57 Nicholson K.G., Aoki F.Y., Osterhaus A.D.M.E. et al. Efficacy and safety of oseltamivir in treatment of acute influenza: a randomised controlled trial. Lancet 2000; 355: 1845-1850.
• 58 de Flora S., Grassi C., Carati L. Attenuation of influenza-like symptomatology and improvement of cell-mediated immunity with long-term N-acetylcysteine treatment. Eur Respir J 1997; 10: 1535-1541.
• 59 Egger M., Smith G.D., Philips A.N. Meta-analysis: principles and procedures. BMJ 1997; 315: 1533-1537.
• 60 Shang A., Huwiler-Muntener K., Martey L., Jüni P., Dörig S., Sterne J.A.C. et al. Author's reply. Lancet 2005; 366: 2083-2085.
• 61 Vergouwen M.D., de Haan R.J., Vermeulen M., Roos Y.B. Statin treatment of hemorrhagic stroke in patients with a history of cerebrovascular disease. Stroke 2008; 39: 497-502.