An Exact Test for Meta-analysis with Binary Endpoints

Summary

Objectives : We reintroduce an exact Mantel-Haenszel (MH) procedure for meta-analysis with binary endpoints which is expected to workespeciallywell i sparse data, e.g., in meta-analyses of safety or adverse events.

Methods : The performance of the exact MH procedure in terms of empirical size and power is compared to the asymptotic MH and to the two standard procedures (fixed effects and random effects model) in a simulation study. We illustrate the methods with a metaanalysis of postoperative stroke occurrence after offpump or on-pump surgery in coronary artery bypass grafting.

Results : We find that in almost all situations the asymptotic MH procedure outperforms its competitors; especially the standard methods yield poor results in terms of power and size.

Conclusions : There is no need to use the reintroduced exact MH procedure; the asymptotic MH procedure will be sufficient in most practical situations. The standard methods (fixed effects and random effects model) should not be used in the sparse data situation.

Keywords

Meta-analysis - odds ratio - logistic models - exact test - SAS®

• References

• 1. Boissel JP, Sacks HS, Leizorovicz A, Blanchard J, Panak E, Peyrieux JC. Meta-analysis of clinical trials: summary of an international conference. Eur J Clin Pharmacol 1988; 34 (06) 535-538.
• 2. Sutton AJ, Abrams KR, Jones DR, Sheldon TA, Song F. Methods for Meta-Analysis in Medical Research. Chichester: Wiley & Sons; 2000
• 3. Whitehead A. Meta-analysis of controlled clinical trials. Chichester: Wiley & Sons; 2002
• 4. Ziegler S, Koch A, Victor N. Deficits and remedy of the standard random effects methods in metaanalysis. Methods Inf Med 2001; 40 (02) 148-155.
• 5. Hartung J, Knapp G. A refined method for the meta-analysis of controlled clinical trials with binary outcome. Stat Med 2001; 20 (24) 3875-3889.
• 6. Biggerstaff BJ, Tweedie RL. Incorporating variability in estimates of heterogeneity in the random effects model in meta-analysis. Stat Med 1997; 16 (07) 753-768.
• 7. Follmann DA, Proschan MA. Valid inference in random effects meta-analysis. Biometrics 1999; 55 (03) 732-737.
• 8. Knapp G, Hartung J. Combinedtest procedures in the meta-analysis of controlled clinical trials. Stud HealthTechnol Inform 2000; 77: 34-38.
• 9. Böhning D. Meta-analysis-Aumfyingmeta-likelihood approach framing unobserved heterogeneity, study covariates, publication bias, and study quality. Methods Inf Med 2005; 44 (01) 127-135.
• 10. Mantel N, Haenszel W. Statistical Aspects of the Analysis of Data from Retrospective Studies of Disease. J Natl Cancer 1 1959; 22 (04) 719-748.
• 11. Emerson JD. Combining estimates of the odds ratio: the state ofthe art. Stat Meth Med Res 1994; 3 (02) 157-178.
• 12. Agresti A. A survey of exact inference for contingency tables (with discussion). Stat Sci 1992; 7 (01) 131-177.
• 13. Sutton AJ, Cooper NJ, Lambert PC, Jones DR, Abrams KR, Sweeting MJ. Meta-analysis of rare and adverse event data. Expert Rev Pharm Out 2002; 2 (04) 367-379.
• 14. Birch MW. Maximum-Likelihood in 3-Way Contingency-Tables. JRSS, Series B 1963; 25 (01) 220-233.
• 15. Vollset SE, Hirji KF, Elashoff RM. Fast Computation of Exact Confidence-Limits for the Common Odds Ratio in a Series of 2x2 Tables. JASA 1991; 86 (414) 404-409.
• 16. Sankey SS, Weissfeld LA, Fine MJ, Kapoor W. An assessment of the use of the continuity correction for sparse data in meta-analysis. Commun Stat - Simul C 1996; 25 (04) 1031-1056.
• 17. Deeks J, Bradburn M, Localio R, Berlin J. Much Ado About Nothing: Statistical Methods for Meta-analysis with Rare Events. Oxford, UK: Centre for Statistics in Medicine, Institute of Health Sciences; 1999
• 18. Sweeting MJ, Sutton AJ, Lambert PC. What to add to nothing? Use and avoidance of continuity corrections inmeta-analysis of sparse data. Stat Med 2004; 23 (09) 1351-1375.
• 19. Mehta CR, Walsh SJ. Comparison of Exact, Mid-P, and Mantel-Haenszel Confidence-Intervals for the Common Odds Ratio Across Several 2 X 2 Contingency-Tables. Am Stat 1992; 46 (02) 146-150.
• 20. Lancaster HO. Significance tests in discrete distributions. JASA 1961; 56 (294) 223-234.
• 21. van Houwelingen HC, Arends LR, Stijnen T. Advanced methods in meta-analysis: multivariate approach and meta-regression. Stat Med 2002; 21 (04) 589-624.
• 22. Cheng DC, Bainbridge D, Martin JE, Novick RJ. Evidence-Based Perioperative Clinical Outcomes Research Group. Does off-pump coronary artery bypass reduce mortality, morbidity and resource utilization when compared with conventional cor onary artery bypass? A meta-analysis of randomized trials. Anesthesiology 2005; 102 (01) 188-203.
• 23. Senn S. The many modes of meta. Drug Inf J 2000; 34 (02) 535-549.
• 24. Breslow N. Odds Ratio Estimators When the Data Are Sparse. Biometrika 1981; 68 (01) 73-84.
• 25. Platt RW, Leroux BG, Breslow N. Generalized linear mixed models for meta-analysis. Stat Med 1999; 18 (06) 643-654.
• 26. Gao S. Combining binomial data using the logistic normal model. J Stat Comput Simul 2006; 74 (04) 293-306.
• 27. Senn S. Controversies concerning randomization and additivity in clinical trials. Stat Med 2004; 23 (24) 3729-3753.
• 28. Greenland S, Robins JM. Estimation of a common effect parameter from sparse follow-up data. Biometrics 1985; 41 (01) 55-68.
• 29. Lui KJ. A Monte Carlo evaluation of five interval estimators for the relative risk in sparse data. Biometrical J 2006; 48 (01) 131-143.
• 30. Hirji KF, Mehta CR, Patel NR. Computing distributions for exact logistic regression. JASA 1987; 82: 1110-1117.