Applying the Jackknife Method to Significance Tests of Diagnostic Diversity

Abstract:

The sample theory of normal diversity indices is complex. Distributionfree methods, such as the jackknife method, can easily be used to determine confidence intervals and testing diversity. Jackknife estimates and their variances for a number of different diversity indices are described in this paper. A simple numerical example is given for demonstrating this method. Discrimination based on confidence intervals is also discussed. It is assumed that there is a special correlation between the sensitivity parameter m and the relative width of confidence intervals in the Hurlbert index family. It is shown that the usual estimation of the Hurlbert index coincides with the relating jackknife estimate. For demonstration, diagnoses registered in a set of death certificates are used. There is a considerable diversity in diagnoses among different diagnostic groups: the diversity is largest in autopsy reports, whereas it is non-significant in GP’s reports and in reports of physicians authorized to issue death certificates. Knowing that autopsy reports tend to be fairly accurate, our research findings seem to confirm the hypothesis that there is a correlation between reliability and diversity of diagnoses.

Keywords

Jackknife - Diversity Indices - Death Causes - Diagnostic Vocabulary

• REFERENCES

• 1 Izsák J. Untersuchungen zur epidemiologischen Diversität. Zeitschrift für die gesamte Hygiene und ihre Grenzgebiete 1989; 35: 223-1.
• 2 Izsák J. Hiding mechanisms in populations and the entropy maximum of death causes: are they related?. Abstracta Botanica 1991; 15: 37-42.
• 3 Izsák J. Secular changes of the concentration of neoplasm death causes in the male population of some countries. Genus 1988; 44: 119-30.
• 4 Farley DE. Case-mix specialization and the concentration of diagnoses in hospitals using information theory. J Health Economics 1989; 08: 185-207.
• 5 Izsák J, Karolyi Gy. Concentration analysis of diagnostic groups in a geronto-epidemiological survey (in Hungarian, with an English abstract). Statisztikai Szemle 1991; 69: 139-16.
• 6 Kempton RA. The structure of species abundance and measurement of diversity. Biometrics 1979; 35: 307-21.
• 7 Rao CR, Boudreau R. Diversity and cluster analyses of blood group data on some human populations. In: Chakravarti ed. Human Population Genetics: The Pittsburg Symposium. New York: Van Nostrand Reinhold Comp; 1984: 331-62.
• 8 Anscombe FJ. Sampling theory of the negative binomial and logarithmic series distributions. Biometrica 1950; 37: 358-82.
• 9 Hutcheson K. A test for comparing diversities based on the Shannon formula. J Theor Biol 1970; 29: 151-4.
• 10 Smith W, Grassle IF. Sampling properties of a family of diversity measures. Biometrics 1977; 33: 283-92.
• 11 Nayak TK. On diversity measures based on entropy functions. Commun Statist Theor Meth 1985; 14: 203-15.
• 12 Kempton RA, Wedderburn RWM. A comparison of three measures of species diversity. Biometrics 1978; 34: 25-37.
• 13 Schuchany WR, Sheather SJ. Jackknifing R-estimators. Biometrika 1989; 76: 393-8.
• 14 Quenouille MH. Notes on bias in estimation. Biometrika 1956; 43: 353-60.
• 15 Tukey JW. Bias and confidence in not quite large samples (abstract). Ann Math Statist 1958; 29: 614.
• 16 Miller RG. The jackknife - a review. Biometrika 1974; 61: 1-15.
• 17 Singh K, Liu RY. On the validity of the jack-knife procedure. Scand J Statist 1990; 17: 11-21.
• 18 Heltshe JF, Forrester NE. Statistical evaluation of the jackknife estimate of diversity when using quadrat samples. Ecology 1985; 66: 107-11.
• 19 Wu CFJ. On the asymptotic properties of the jackknife histogram. The Annals of Statistics 1990; 18: 1438-52.
• 20 Mueller LD, Altenberg L. Statistical inference on measures of niche overlap. Ecology 1985; 66: 1204-10.
• 21 Shi X. Some asymptotic results for jackknifing the sample quantile. The Annals of Statistics 1991; 19: 496-503.
• 22 Shao J, Wu CFJ. A general theory for jackknife variance estimation. The Annals of Statistics 1989; 17: 1176-97.
• 23 Frangos CC. An updated bibliography on the jackknife method. Comm Statist A 1987; 16: 1543-84.
• 24 Zahl S. Jackknifing an index of diversity. Ecology 1977; 58: 907-13.
• 25 Routledge RD. Bias in estimating the diversity of large, uncensured communities. Ecology 1980; 06: 276-81.
• 26 Heltshe JF, Bitz DW. Comparing diversity measures in sampled communities. In: Grassle JF, et al. ed. Ecological Diversity in Theory and Practice. Fairland MD: Internat Publishing House; 1979: 133-44.
• 27 Heltshe JF, Forrester NE. Estimating species richness using the jackknife procedure. Biometrics 1983; 39: 1-11.
• 28 Lyons NK, Hutcheson K. Estimation of Simpson’s diversity when counts follow a Poisson distribution. Biometrics 1986; 42: 171-6.
• 29 Adams JE. McCune Application of the generalized jackknife to Shannon’s measure of information used as an index of diversity. In: Grassle JF. et al. ed. Ecological Diversity in Theory and Practice. Fairland MD: Internat Publishing House; 1979: 117-31.
• 30 Karolyi G. Information on morbidity from death certificates and autopsy reports (in Hungarian, with an English abstract). Statisztikai Szemle 1990; 68: 564-77.
• 31 Hurlbert SM. The nonconcept of species diversity: a critique and alternative parameters. Ecology 1971; 52: 577-86.
• 32 Hill MO. Diversity and evenness: a unifying notation and its consequences. Ecology 1973; 54: 427-32.
• 33 Williams CB. Patterns in the Balance of Nature. London - New York: Academic Press; 1964
• 34 Fisher RA, Corbet AS, Williams CB. The relation between the number of species and the number of individuals in a randomsample of an animal population. J Anim Ecol 1943; 12: 42-58.
• 35 Birch MW. An algorithm for the logarithmic series distribution. Biometrics 1963; 19: 651-2.
• 36 Izsák J, Juhász-Nagy P. Studies of lognormality on mortality statistics. Biometrical J 1982; 24: 731-41.
• 37 Basharin GP. On a statistical estimate for the entropy of a sequence of independent random variables. Theory of Probability and its Applications 1959; 04: 333-6.