Evaluation of Adjusted and Unadjusted Indirect Comparison Methods in Benefit Assessment

Summary

Background: With the Act on the Reform of the Market for Medicinal Products (AMNOG) in Germany, pharmaceutical manufacturers are obliged to submit a dossier demonstrating added benefit of a new drug compared to an appropriate comparator. Underlying evidence was planned for registration purposes and therefore often does not meet the appropriate comparator as defined by the Federal Joint Committee (G-BA). For this reason AMNOG allows indirect comparisons to assess the extent of added benefit.

Objectives: The aim of this study is to evaluate the characteristics and applicability of adjusted indirect comparison described by Bucher and Matching-Adjusted Indirect Comparison (MAIC) in various situations within the early benefit assessment according to §35a Social Code Book 5. In particular, we consider time-to-event endpoints.

Methods: We conduct a simulation study where we consider three different scenarios: I) similar study populations, II) dissimilar study populations without interactions and III) dissimilar study populations with interactions between treatment effect and effect modifiers. We simulate data from a Cox model with Wei- bull distributed survival times. Desired are unbiased effect estimates. We compare the power and the proportion of type 1 errors of the methods.

Results: I) Bucher and MAIC perform equiva- lently well and yield unbiased effect estimates as well as proportions of type 1 errors below the significance level of 5%. II) Both Bucher and MAIC yield unbiased effect estimates, but Bucher shows a higher power for detection of true added benefit than MAIC. III) Only MAIC, but not Bucher yields unbiased effect estimates. When using robust variance estimation MAIC yields a proportion of type 1 error close to 5%.

In general, power of all methods for indirect comparisons is low. An increasing loss of power for the indirect comparisons can be observed as the true treatment effects decrease.

Conclusion: Due to the great loss of power and the potential bias for indirect comparisons, head-to-head trials using the appropriate comparator as defined by the Federal Joint Committee should be conducted whenever possible. However, indirect comparisons are needed if no such direct evidence is available. To conduct indirect comparisons in case of a present common comparator and similar study populations in the trials to be compared, both Bucher and MAIC can be recommended. In case of using adjusted effect measures (such as Hazard Ratio), the violation of the similarity assumption has no relevant effect on the Bucher approach as long as interactions between treatment effect and effect modifiers are absent. Therefore Bucher can still be considered appropriate in this specific situation. In the authors’ opinion, MAIC can be considered as an option (at least as sensitivity analysis to Bucher) if such interactions are present or cannot be ruled out. Nevertheless, in practice MAIC is potentially biased and should always be considered with utmost care.

Keywords

Indirect comparisons - Bucher - MAIC - AMNOG - dissimilarity

• References

• 1 Bundesministerium für Justiz und für Verbraucherschutz (2015). Sozialgesetzbuch (SGB) Fünftes Buch (V) - Gesetzliche Krankenversicherung - (Artikel 1 des Gesetzes vom 20. Dezember 1988, BGBl. I S. 2477) §35a Bewertung des Nutzens von Arzneimitteln mit neuen Wirkstoffen: Bundesministerium für Justiz und für Verbraucherschutz (cited 2015 Dec 14). Available from: http://www.gesetze-im-internet.de/sgb_5/35a.html.
• 2 Lu G, Ades AE. Combination of direct and indirect evidence in mixed treatment comparisons. Statistics in Medicine. 2004; 23: 3105-3124.
• 3 Bucher HC, Guyatt GH, Griffith LE, Walter SD. The results of direct and indirect treatment comparisons in meta-analysis of randomized controlled trials. Journal of Clinical Epidemiology. 1997; 50 (06) 683-691.
• 4 IQWiG Institut für Qualität und Wirtschaftlichkeit im Gesundheitswesen (2015). Allgemeine Methoden Version 4.2 vom 22.04.2015 (cited 2016 Dec 23). Available from: https://www.iqwig.de/download/IQWiG_Methoden_Version_4-2.pdf.
• 5 Skipka G, Wieseler B, Kaiser T, Thomas S, Bender R, Windeler J, Lange S. Methodological approach to determine minor, considerable, and major treatment effects in the early benefit assessment of new drugs. Biometrical Journal. 2016; 58: 43-58.
• 6 Kiefer C, Sturtz S, Bender R. Indirect comparisons and network meta-analyses: estimation of effects in the absence of head-to-head trials - part 22 of a series on evaluation of scientific publications. Deutsches Aerzteblatt International. 2015; 112: 803-808.
• 7 Signorovitch JE, Wu EQ, Yu AP, Gerrits CM, Kantor E, Bao Y, Gupta SR, Mulani PM. Comparative Effectiveness Without Head-to-Head Trials: A Method for Matching-Adjusted Indirect Comparisons Applied to Psoriasis Treatment with Adalimumab or Etanercept. Pharmacoeconomics. 2010; 28 (10) 935-945.
• 8 Austin PC. An Introduction to Propensity Score Methods for Reducing the Effects of Confounding in Observational Studies. Multivariate Behavioral Research. 2011; 46: 399-424.
• 9 Glynn RJ, Schneeweiss S, Stuermer T. Indications for Propensity Scores and Review of Their Use in Pharmacoepidemiology. Basic and Clinical Pharmacology and Toxicology. 2006; 98 (03) 253-259.
• 10 Signorovitch JE, Swallow E, Kantor E, Wang X, Klimovsky J, Haas T, Devine B, Metrakos P. Everolimus and Sunitinib for advanced pancreatic neuroendocrine tumors: a matching adjusted indirect comparison. Experimental Hematology Oncology. 2013; 2: 32.
• 11 Bender R, Augustin T, Blettner M. Generating Survival Times to Simulate Cox Proportional Hazards Models. German Research Foundation (DFG), Sonderforschungsbereich 386, Paper 338; 2003 (cited 2015 Aug 11). Available from: https://epub.ub.uni-muenchen.de/1716/1/paper_338.pdf.
• 12 Cox DR. Regression models and life tables (with discussion). Journal of the Royal Statistical Society. 1972; (Series B). 74: 187-200.
• 13 R Core Team. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing.. Vienna: 2014. (cited 2014 Dec 12). Available from: http://cran.r-project.org/.
• 14 Therneau T. A Package for Survival Analysis in S. R package version 2.37-7. 2014 (cited 2014 Dec 12). Available from: http://CRAN.R-project.org/package=survival.
• 15 Qiu W, Chavarro J, Lazarus R, Rosner B, Ma J. powerSurvEpi: Power and sample size calculation for survival analysis of epidemiological studies. R package version 0.0.6. 2012 (cited 2014 Dec 12). Available from: https://cran.r-project.org/web/packages/powerSurvEpi/index.html.
• 16 Venables WN, Ripley BD. Modern Applied Statistics with S.. 4th ed. New York: Springer-Verlag; 2002
• 17 Cummings P. Methods for estimating adjusted risk ratios. The Stata Journal. 2009; 9 (02) 175-196.
• 18 Freedman DA. On The So-Called “Huber Sandwich Estimator” and “Robust Standard Errors” (cited 2015 Sep 22). The American Statistician.. 2006 60 04 Available from: http://www.stat.berkeley.edu/~census/mlesan.pdf.
• 19 Snapinn S, Jiang Q. Indirect comparisons in the comparative efficacy and non-inferiority settings. Pharmaceutical Statistics. 2011; 10: 420-426.
• 20 Ishak KJ, Proskorovsky I, Benedict A. Simulation and Matching-Based Approaches for Indirect Comparison of Treatments. Pharmacoeconomics. 2015; 33: 537-549.
• 21 Caro JJ, Ishak KJ. No Head-to-Head Trial? Simulate the Missing Arms. Pharmacoeconomics. 2010; 28 (10) 957-967.
• 22 Ishak KJ, Rael M, Phatak H, Masseria C, Lanitis T. Simulated Treatment Comparison of Time-to-Event (and Other Non-Linear) Outcomes. ISPOR 2015. 18th Annual European Congress; 2015
• 23 Quintana M, Broglio K, Daar ES, Yuan Y, Kalsekar A, Detry M, Le T, Berry SM. Determining the Comparative Effectiveness of Emerging Treatment Regimens for Hepatitis C Virus (HCV) Infection from Single Arm Phase III Trials. EASL 2014. 49th Annual Meeting; 2014
• 24 Peters JL, Rushton L, Sutton AJ, Jones DR, Abrams KR, Mugglestone MA. Bayesian Methods for the Cross-Design Synthesis of Epidemiological and Toxicological Evidence. Journal of the Royal Statistical Society. 2005; Series C (Applied Statistics) 5 (01) 159-172.
• 25 Schmitz S, Adams R, Walsha C. Incorporating data from various trial designs into a mixed treatment comparison model. Statistics in Medicine. 2013; 32: 2935-2949.