Download PDF
Homeopathy 2010; 99(02): 148-149
DOI: 10.1016/j.homp.2010.02.004
Letter to the Editor
Copyright © The Faculty of Homeopathy 2010

Clinical trials, clinical evidence, and selective citation

Authors

  • Leslie Rose

  • John Garrow


Subject Editor:
Further Information

Publication History

Publication Date:
16 December 2017 (online)

Also available at
   Permissions and Reprints
Preview

Dr Lionel Milgrom's reliance on the authority of Sir Michael Rawlins may be misplaced.[ 1,2 ] To give the quotation its proper context, Sir Michael actually said this about randomised controlled trials (RCTs) in his Harveian Oration of October 2008:

“The technique has three great advantages. It minimizes bias, especially selection bias because every patient has an equal chance of receiving each of the treatments. It minimizes confounding factors because as a result of randomization these are likely to be evenly distributed among the groups. And provided the groups are of an appropriate size it minimizes the play of chance or random error as statisticians like to call it”. [ 3 ]

Shortly after that event, he said:

“As far as homeopathy is concerned it breaks every rule in the evidential base! It is biologically implausible; it is almost always used to treat conditions where the natural history is unpredictable; and the signal to noise ratio is close to one!” [ 4 ]

Perhaps more importantly, Milgrom misunderstands the now infamous pie chart published by BMJ Clinical Evidence. What the chart does not tell us is what proportion of patients receive a treatment that is ‘beneficial’. Unfortunately we are not told how many diseases are covered in this survey. If we look up the advice on a given condition (eg. low back pain) there are 18 treatments considered, but probably for other less common diseases there will be fewer treatments considered. Let us be generous and assume that on average there are 10 treatments per disease, then 2500 treatments are being used for 250 diseases. But we know that there are at least 12% of beneficial treatments, so 12% of 2500 is 300, so the data are compatible with 100% of patients having at least one effective treatment, with a spare 50 treatments for a second choice. What we need (and should try to collect) is the evidence for the treatments that patients actually receive.

Indeed that has already been done for several specialisms. Several published studies put the proportion of evidence based clinical practice (not treatments) at between 55% and 97%.[ 5–15 ] Not good enough, but far better than the Clinical Evidence pie chart suggests. In any case we are unconvinced by the ‘tu quoque’ argument.

Milgrom also misunderstands the role of the RCT. We know of no-one who is advocating the ‘monoculture’ he describes. Sir Michael Rawlins was arguing quite rightly for a more inclusive approach to evidence, in particular because we need to know whether RCT evidence is generalisable to normal clinical practice. He did not say that RCTs should routinely be replaced by less rigorous methodologies. It may be tempting for chiropractors and homeopaths to rely on uncontrolled observational data when RCTs fail to give them the answers they want, but would that be good science?