A Note on Testing for Intervention Effects on Binary Responses

Summary

Objectives: In some circumstances controlled trials are not feasible and treatments can only be evaluated using clinical databases. Here we consider the situation where treatment is introduced at a particular calendar time and can only be evaluated by comparison with historical controls. In these circumstances Heuer and Abel recommended using change-point methods to search for change in characteristics over the whole study period rather than simply comparing treated and untreated patients. Their recommendation is to only conclude that the intervention had an effect if a change-point could be demonstrated close in time to the introduction of the new treatment. This reduces the risk of false positives caused by confounding changes in population characteristics or changes in patient management. For binary data we develop a method that follows their philosophy and apply it to an observational study in the treatment of pin sites after orthopaedic surgery.

Methods: Tests for change in binomial probabilities based on Brownian bridge and Hansen’s approximation for maximally selected X ² statistics are compared to an exact test by Worsley. The approximate method is generalized to logistic regression models allowing for covariates.

Results: The agreement of the exact and approximate method is good for sample sizes of 100 or more. The actual test size of the Hansen approximate test allowing for covariates is close to the nominal level, whereas the Brownian bridge approximation is slightly conservative. The change in pin site treatment significantly reduces the risk of infection for both adults and children.

Conclusions: We consider the Hansen approximation to provide a very good and very simple method for obtaining the p-value when testing for a change in binary data event probabilities, with or without covariates.

Keywords

Change-point - observational study - logistic regression - pin site infection data

• References

• 1 Friede T, Henderson R. Intervention effects in observational studies, with an application in total hip replacements. Statistics in Medicine 2003; 22: 3725-37.
• 2 Heuer C, Abel U. The analysis of intervention effects using observational data bases. In Abel U, Koch A. (eds.) Nonrandomized comparative clinical studies. Duesseldorf: Symposium Publishing 1998; pp 101-7.
• 3 Page ES. Continuous inspection schemes. Biometrika 1954; 41: 100-15.
• 4 Page ES. A test for a change in a parameter occurring at an unknown point. Biometrika 1955; 42: 523-7.
• 5 Sen A, Srivastava MS. On tests for detecting change in mean. Annals of Statistics 1975; 3: 98-108.
• 6 Worsley KJ. The power of likelihood ratio and cumulative sum tests for a change in a binomial probability. Biometrika 1983; 70: 455-64.
• 7 Worsley KJ. Confidence regions and tests for a change-point in a sequence of exponential family variables. Biometrika 1986; 73: 91-104.
• 8 Hinkley DV. Inference about the change-point in a sequence of random variable. Biometrika 1970; 57: 1-17.
• 9 Hinkley DV, Hinkley EA. Inference about the change-point in a sequence of binomial random variables. Biometrika 1970; 57: 477-88.
• 10 Yao YC. Approximating the distribution of the maximum likelihood estimate of the change-point in a sequence of independent random variables. Ann Statist 1987; 15: 1321-8.
• 11 Siegmund D. Boundary crossing probabilities and statistical applications. Annals of Statistics 1986; 14: 361-404.
• 12 Chernoff H, Zacks S. Estimating the current mean of a normal distribution which is subjected to a change in time. Annals of Mathematical Statistics 1964; 35: 999-1018.
• 13 Smith AFM. A Bayesian approach to inference about a change-point in a sequence of random variables. Biometrika 1975; 62: 407-16.
• 14 Bhattacharya PK, Frierson D. A nonparametric chart for detecting small disorders. Annals of Statistics 1981; 9: 544-54.
• 15 Carlstein E. Nonparametric change-point estimation. Annals of Statistics 1988; 16: 188-97.
• 16 Zacks S. Classical and Bayesian approaches to the change-point problem: fixed sample and sequential procedures. Statistique et Analyse des Données 1982; 1: 48-81.
• 17 Henderson R, Matthews JNS. An investigation of changepoints in the annual number of cases of hemolytic-uremic syndrome. Applied Statistics 1993; 42: 461-71.
• 18 Henderson R. Change-point problem with correlated observations, with an application in material accountancy. Technometrics 1986; 28: 381-9.
• 19 Kim HJ. Change-point detection for correlated observations. Statistica Sinica 1996; 6: 275-87.
• 20 Matthews DE, Farewell VT. On a singularity in the likelihood for a change-point hazard rate model. Biometrika 1985; 72: 703-4.
• 21 Henderson R. A problem with the likelihood ratio test for a change-point hazard rate model. Biometrika 1990; 77: 835-43.
• 22 Gijbels I, Gurler U. Estimation of a change point in a hazard function based on censored data. Lifetime Data Analysis 2003; 9: 395-411.
• 23 Miller R, Siegmund D. Maximally selected chi square statistics. Biometrics 1982; 38: 1011-6.
• 24 Halpern J.. Maximally selected chi square statistics for small samples. Biometrics 1982; 38: 1017-23.
• 25 Davies RB. Hypothesis testing when a nuisance parameter is present only under the alternative. Biometrika 1987; 74: 33-43.
• 26 Andrews DA, Ploberger W. Optimal tests when a nuisance parameter is present only under the alternative. Econometrica 1994; 62: 1383-1414.
• 27 Betensky R, Rabinowitz D. Maximally selected χ² statistics for k × 2 tables. Biometrics 1999; 55: 317-20.
• 28 Halpern AL. Minimally selected p and other tests for a single abrupt changepoint in a binary sequence. Biometrics 1999; 55: 1044-5.
• 29 Hansen BE. Approximate asymptotic p values for structural-change tests. Journal of Business & Economic Statistics 1997; 15: 60-7.
• 30 Lausen B, Schumacher M. Maximally selected rank statistics. Biometrics 1992; 48: 73-85.
• 31 Lausen B, Hothorn T, Bretz F, Schumacher M.. Assessment of optimal selected prognostic factors. Biometrical Journal 2004; 46: 364-74.
• 32 Halpern AL. Multiple-changepoint testing for an alternating segments model of a binary sequence. Biometrics 2000; 56: 903-8.