Genomic Profiling

Summary

Objectives: The current literature on the use of micro-arrays to generate prognostic profiles is still a methodological wasteland. Many valid questions, such as how profiling studies should be evaluated and what conclusions can be drawn, remain unanswered. Or how can flaws in the collection, analysis and interpretation of data be detected? Undiscovered imperfections can lead to a waste of valuable resources as well as be a latent source of false conclusions.

Methods: Three seminal papers on the prognosis of breast cancer using genomic profiling will be discussed. Six principles of good experimental design will be used for methodological guidance: defining relevant endpoints, avoiding systematic bias, generalizability of results, appropriately sized samples to achieve sufficient power, simple design to improve interpretability, and avoiding artificial assumptions.

Results: Severe violations of at least one of the six principles of good experimental design can be found in each of the three papers. A strategy is presented to assess whether a study has achieved a high level of methodological quality. This strategy also helps to establish a suitable protocol for future profiling projects.

Conclusions: Determining the design of a study in a protocol is a first step to avoid impending pitfalls. The protocol should deal with the problem of understanding the complex reality behind genomic profiling. There are basic guiding principles which can help handle the complex task of designing prognostic studies to find genomic signatures.

Keywords

Microarray profiling - cancer - experimental design - clinical epidemiology - computational statistics

• References

• 1 Simon R, Altman D. Statistical aspects of prognostic factor studies in oncology. Br J Cancer 1994; 69: 979-85.
• 2 van ’t Veer L, Dai H, van de Vijver MJ, He YD, Hart AAM, Mao M, Peterse HL, van de Kooy K, Marton MJ, Witteveen AT, Schreiber GJ, Kerkhoven RM, Roberts C, Linsley PS, Bernards R, Friend SH. Gene expression profiling predicts clinical outcome of breast cancer. Nature 2002; 415: 530-6.
• 3 van de Vijver MJ, He YD, van ’t Veer LJ, Dai H, Hart AA, Voskuil DW, Schreiber GJ, Peterse JL, Roberts C, Marton MJ, Parrish M, Atsma D, Witteveen A, Glas A, Delahaye L, van der Velde T, Bartelink H, Rodenhuis S, Rutgers ET, Friend SH, Bernards R. A gene-expression signature as a predictor of survival in breast cancer. N Engl J Med 2002; 347: 1999-2009.
• 4 Huang E, Cheng SH, Dressman H, Pittman J, Tsou MH, Horng CF, Bild A, Iversen ES, Liao M, Chen CM, West M, Nevins JR, Huang AT. Gene expression predictors of breast cancer outcomes. The Lancet 2003; 361: 1590-6.
• 5 Chang JC, Wooten EC, Tsimelzon A, Hilsenbeck SG, Gutierrez MC, Elledge R, Mohsin S, Osborne CK, Chamness GC, Allred DC, O’Connell P. Gene expression profiling for the prediction of therapeutic response to docetaxel in patients with breast cancer. The Lancet 2003; 362: 362-9.
• 6 Moher D, Schulz KF. Altman DG for the CONSORT Group. The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomised trials. The Lancet 2001; 357: 1191-4.
• 7 Moher D, Cook DJ, Eastwood S, Olkin I, Rennie D. Stroup DF for the QUOROM Group. Improving the quality of reports of meta-analyses of randomized controlled trials: the QUOROM Statement. The Lancet 1999; 354: 1896-1900.
• 8 Bossuyt PM, Reitsma JB, Bruns DE, Gatsonis CA, Glasziou PP, Irwig LM, Lijmer JG, Moher D, Rennie D, de Vet HC. STARD group. Towards complete and accurate reporting of studies of diagnostic accuracy: the STARD initiative. Clin Chem Lab Med 2003; 41: 68-73.
• 9 Rothman KJ, Greenland S. Modern Epidemiology. Philadelphia: Lippincott; 1997
• 10 Tibshirani RJ, Efron B. Pre-validation and inference in microarrays. Statistical Applications in Genetics and Molecular Biology 2002; 1: 1
• 11 Cox DR. Planning of experiments. New York: Wiley & Sons; 1958
• 12 Brenton JD, Caldas C. 2003 Predictive cancer genomics – what do we need?. The Lancet 2003; 362: 340-1.
• 13 Faneyte IF. et al. Breast cancer response to neo-adjuvant chemotherapy: predictive markers and relation with outcome. Br J Cancer 2003; 88: 406-12.
• 14 Markowetz F, Spang R. Molecular Diagnosis: Classification, Model Selection and Performance Evaluation. Methods Inf Med 2005; 44: 438-43. (this issue)
• 15 Ambroise C, McLachlan GJ. Selection bias in gene extraction on the basis of microarray gene expression data. Proc Natl Acad Sci 2003; 99: 6562-6.
• 16 Pepe MS, Longton G, Anderson GL, Schummer M. Selecting differentially expressed genes from microarray experiments. Biometrics 2003; 59: 133-42.
• 17 Ruschhaupt M, Huber W, Poustka A, Mansmann U. A Compendium to Ensure Computational Reproducibility in High-Dimensional Classification Tasks. Statistical Applications in Genetics and Molecular Biology 2004; 3 (01) article 37 http://www.bepress.com/sagmb/vol3/iss1/art37
• 18 http://www.geneontology.org
• 19 Ramaswamy S, Perou CM. DNA microarray in breast cancer: The promise of personalized medicine. The Lancet 2003; 361: 1576-7.
• 20 Breit S, Nees M, Schaefer U, Pfoersich M, Hagemeier C, Muckenthaler M, Kulozik A. Impact of pre-analytic handling on bone marrow mRNA gene expression. BJH 2004; 126: 231-43.
• 21 Goeman JJ, van de Geer SA, de Kort F, van Houwelingen HC. A global test for groups of genes: testing association with a clinical outcome. Bioinformatics 2004; 20: 93-9.
• 22 Mansmann U, Meister R. Testing differential gene expression in functional groups: Goeman’s global test versus an ANCOVA approach. Methods Inf Med 2005; 44: 449-53. (this issue)
• 23 Schmoor C, Sauerbrei W, Schumacher M. Sample size considerations for the evaluation of prognostic factors in survival analysis. Stat Med 2000; 19: 441-52.
• 24 Leisch F, and Rossini AJ. Reproducible Statistical research. Chance 2003; 16: 41-5.
• 25 Sawitzki G. Keeping Statistics Alive in Documents. Computational Statistics 2002; 17: 65-88.
• 26 http://www.biometrie.uni-heidelberg.de/mb/SASmakros.htm
• 27 www.r-project.org
• 28 www.bioconductor.org
• 29 Bogardus Jr ST, Concato J, Feinstein AR. Clinical epidemiological quality in molecular genetic research: the need for methodological standards. JAMA 1999; 28: 1919-26.