Response to “Rates of Inhibitor Development in Previously Untreated Patients with Severe Hemophilia A Treated with Plasma-Derived or Recombinant Factor VIII: No Proof of Difference or Proof of No Difference?”
The issue of comparing the rates of inhibitor development in previously untreated patients with severe hemophilia A who are treated with plasma-derived Factor VIII concentrates (pdFVIII) or recombinant Factor VIII (rFVIII) is influenced by numerous and unavoidable biases. These biases essentially depend on the observational nature of the data and on the recognition that factors other than the type of product might be implicated. In this framework, the meta-analysis by Franchini et al[1 ] represents the most comprehensive and updated attempt to shed light on this controversial topic.
In the discussion of the results of this meta-analysis, one key of interpretation has not been considered by the authors but deserves, in our view, to be pointed out. The question essentially is: Do these results represent no proof of difference or proof of no difference?
When a traditional meta-analysis generates a nonsignificant result, the need to differentiate between no proof of difference or proof of no difference is increasingly being recognized[2 ]
[3 ]
[4 ]
[5 ]; in fact, in the first case the results should be viewed as inconclusive (i.e., no demonstration of anything) while in the second case a statistical proof is obtained that no difference exists (i.e., demonstration of equivalence or noninferiority).
To address this specific question, we have reanalyzed the results described by Franchini et al by carrying out a formal equivalence test based on 95% confidence intervals (CIs). The objective was to compare the inhibitor development rates between pdFVIII and rFVIII; the main assumption of our analysis was that the margins for equivalence are set at 0 ± 4.0%.
[Fig. 1 ] summarizes our reanalysis. To determine the 95% CIs shown in [Fig. 1 ], we first repeated the proportion meta-analysis of Franchini et al and we exactly confirmed the meta-analytic results published by the Italian authors (details on our repeated meta-analysis with study-specific crude rates and Forest plots can be seen from our institutional Web site at http://www.sifact.it/firenze/sth-messori-supplement.pdf ); confirmation of the crude rates was a prerequisite for undertaking our equivalence analysis. The main result of our equivalence analysis was that the pooled rate difference of +6% for rFVIII versus pdFVIII (i.e., the difference between the two point estimates of 29 vs. 23%, respectively) is associated to a standard error of 5.335% and to a 95% CI ranging from –4.46 to +16.46%. According to our reanalysis, the published data reported by the Italian authors clearly represent no proof of difference.
Fig. 1 Rates of inhibitor development in previously untreated patients with severe hemophilia A: test of equivalence applied to recombinant Factor VIII in comparison with plasma-derived Factor VIII. The graph is based on the same primary data as those reported by Franchini et al[1 ] in their Table 2. The absolute risk was 23% (95% confidence interval [CI], 15–33%) in patients treated with plasma-derived concentrates and 29% (95% CI, 26–32%) in those treated with recombinant Factor VIII. The equivalence test is based on the area comprised between the two red vertical lines, that reflect the predetermined equivalence margins (around the value of rate difference = 0 ± 4.0%). The horizontal black line indicates the two-sided 95% CI (or one-sided 97.5% CI) for the pooled value of rate difference between rFVIII and pdFVIII found by Franchini et al (point estimate represented by the solid square); the estimation of this 95% CI, which is the crucial step in equivalence testing, was performed according to the methods described by Ahn et al[6 ] and Hausleiter et al.[7 ] Accordingly, the standard error for the pooled rate difference was first estimated (standard error = 5.335%) and then the lower and upper extremes for this 95% CI were determined (–4.46 and +16.46%, respectively).
More importantly, our results show that these data do not support the conclusion of proof of no difference (i.e., equivalence) unless one extends the definition of equivalence up to an unrealistic margin of 0 ± 17.0%. Another unsettled question is that the need to differentiate between different products containing rFVIII further increases the difficulty in studying this topic.
We conclude that, despite the inclusion of the best evidence currently available, we still do not know whether or not rFVIII is associated to an increased rate of inhibitor development as compared with pdFVIII. Finally, while we admit that most of the considerations presented herein were implicit in the article by Franchini et al, we think that making these points more explicit is worthwhile.
