Weighing the homeopathic evidence

This issue of ‘Homeopathy’ contains an article[1] which is in several ways remarkable and deserves a comment. Essentially it is an attempt to take a fresh look at the homeopathic trial evidence.

A remarkable feature is the methodological approach of the analysis. Robert Mathie evaluated the ‘weight’ of the evidence, a term not usually used in this context. Having said this, we have used it extensively in our recent assessment of the entire field of complementary medicine.[2] We employed the term as a composite measure of three independent variables associated with any trial evidence (i) methodological quality, (ii) volume (ie total sample size), (iii) level (ie uncontrolled, controlled or randomised trial, systematic review or meta-analysis). Mathie's used the term ‘weight’ to describe a simple ‘vote count’, ie negative vs positive trials, and he attributed a positive weight if the latter outnumbered the former by at least two.

The most remarkable feature of the article is its overall result. It concludes that ‘the present weight of evidence favours homoeopathic treatment efficacy in 8 conditions’. This overtly contradicts most of the 17 systematic reviews that have evaluated (some of) this evidence since 19973 and contradicts my review of these reviews, which concluded that ‘the best clinical evidence for homeopathy available to date does not warrant positive recommendations for its use in clinical practice’.[3]

How can this be? Surely systematic reviews are supposed to minimise bias! Should they therefore not reach identical conclusions? Linde and Willich have recently suggested that this is not necessarily the case and that ‘systematic reviews often differ considerably’.[4] They demonstrate that different methodologies are the cause of such discrepancies.

We therefore need to ask how good Mathie's methodology was and how sound his conclusions are. The eight conditions for which he finds positive evidence are listed in [Table 1]. In five cases (influenza, childhood diarrhoea, fibrositis, sprains and radio/chemotherapy), there are only two or three trials available; in some cases these studies are small. Thus, the volume of the evidence is clearly not substantial. In other instances there is a clear need for independent replication of the trials, as the data originate exclusively from only one or two research groups. In yet other cases, the methodological quality of some of the studies is such that it seriously limits the conclusiveness of the result.

Table 1 Critique of positive trials

Condition	Vote count	Caveats
Hay fever/allergic rhinitis	6 : 3	Three trials were on isopathy rather than homoeopathy, all of the six positive trials were from only two research groups
Influenza	2 : 0	Small volume of evidence
Upper respiratory tract infection	7 : 3	Several trials of very poor methodological quality
Childhood diarrhoea	3 : 0	Small volume of evidence, all trials from one research group
Fibrositis/fibromyalgia	2 : 0	Small volume of evidence
Sprains	2 : 0	Small volume of evidence
Pain	5 : 2	Extremely heterogeneous group of conditions, some trials of poor methodological quality
Radiotherapy/chemotherapy (side effects)	3:0	Small volume of evidence

But there are other problems as well. For instance, Mathie included studies that were ‘randomised and/or double-blind’. In other words, non-randomised trials were included, which are known to be wide open to selection bias. Mathie also categorised a trial result as positive if ‘at least one outcome measure was statistically significantly improved’. Consider a trial with 20 outcome measures and a statistical cutoff at P = 0.05. This is by no means a rare scenario’a study with five endpoints measured at four different timepoints would already fulfil this criterion. In such a case, one outcome measure would normally be significant purely by chance rather than through anything remotely associated with therapeutic efficacy. The more rigorous criterion for a ‘positive’ trial is thus that the endpoint, which had been predefined by the original authors as the primary outcome measure, yielded significant results.

Thus, I fear, Mathie's methodology was not as strong as it could have been, and this deficit has led to conclusions that may be falsely positive. People who are less enthusiastic about science and more emotionally dedicated to homoeopathy than I am will emphatically dispute this point’at least I hope they will. It is this dispute that will drive us to conduct more and better trials; and it is only through such tests that, one day perhaps, we will be able to document the truth about homoeopathy's value beyond placebo.

Publication History

Article published online:
16 September 2021

• References

• 1 Mathie RT. The research evidence base for homeopathy: a fresh assessment of the literature. Homeopathy 2003; 92: 80-87
• 2 Ernst E, Pittler MH, Stevinson C, White AR, Eisenberg D. The Desktop Guide to Complementary and Alternative Medicine. Edinburgh: Mosby; 2001
• 3 Ernst E. A systematic review of systematic reviews of homeopathy. Br J Clin Pharmacol 2002; 54: 577-582
• 4 Linde K, Willich SN. How objective are systematic reviews? Differences between reviews on complementary medicine. J R Soc Med 2003; 96: 17-22