A Bayesian Approach to Estimate and Validate the False Negative Fraction in a Two-stage Multiple Screening Test

Summary

Objectives: In estimating sensitivity and specificity of a diagnostic kit it is imperative that all study subjects are verified via a gold standard procedure. However the application of such a procedure to all the study subjects may not be feasible due to associated cost, risk and invasiveness. As a result only a part of the study subjects receive the definitive assessment. The accuracy of a diagnostic kit can also be expressed in terms of its error rates. Our first objective is to estimate the false negative fraction (FNF) under partial verification in a particular case of a two-stage multiple screening test using a beta-binomial model and a Bayesian logistic model. The second objective is to validate the two models in order to determine which fits the data better.

Methods: We estimate the FNF from the above mentioned models using Bayesian approach. The validation of the models is based on their out-of-sample predictive capabilities.

Results: For the bowel cancer data that was used in this study we found the median posterior estimate of the FNF, based on the beta-binomial model, to be 26.4% (95% credible interval: 0.123-0.650). The corresponding estimate based on the Bayesian logistic model was 23.3% (95% credible interval: 0.124-0.375). Validation results showed that the beta-binomial model gave slightly better predictions compared to the Bayesian logistic model.

Conclusions: Estimation of the FNF can be done by adopting the Bayesian approach. Models fitted can be validated by comparing their performance in terms of their out-of-sample predicitve potential.

Keywords

Bayesian analysis - MCMC - prediction - sensitivity - validation

• References

• 1 Begg CB. Assessment of diagnostic tests when disease verification is subject to selection bias. Biometrics 1983; 39: 207-15.
• 2 Walter SD. Estimation of test sensitivity and specificity when disease confirmation is limited to positive results. Epidemiology 1999; 10: 67-72.
• 3 Lloyd CJ, Frommer DJ. Estimating the false negative fraction for a multiple screening test for bowel cancer when the negatives are not verified. Austral NZ J Statist. 2004. (in press)
• 4 Lloyd CJ, Frommer DJ. Regression based estimation of the false negative fraction when multiple negatives are unverified. Appl Statist 2004; 53: 619-31.
• 5 O’Hagan A. HSSS model criticism. In Green PJ, Hjort NL, Richardson S. eds Highly Structured Stochastic Systems. Oxford: Oxford University Press; 2003: 423-44.
• 6 Lee JC, Sabavala DJ. Bayesian estimation and prediction for the beta-binomial model. J Busin and Econ Statist 1987; 5: 357-67.
• 7 Williams DA. The analysis of binary responses from toxicological experiments involving reproduction and teratogenicity. Biometrics 1975; 31: 949-52.
• 8 McCullagh P, Nelder JA. Generalized linear models. London: Chapman and Hall; 1989
• 9 Lloyd CJ. Analysis of categorical data. New York: Wiley; 1999
• 10 Bernardo JM, Smith AFM. Bayesian theory. New York: Wiley; 1994
• 11 van der Linde A. Personal communication. 2003
• 12 Hastings WK. Monte Carlo sampling methods using Markov chains and their applications. Biometrika 1970; 57: 97-109.
• 13 Gamerman D. Sampling from the posterior distribution in generalized linear mixed models. Statist and Comp 1997; 7: 57-68.
• 14 Gelman A, Roberts GO, Gilks WR. Efficient Metropolis jumping rules. In Bernardo JM, Berger J, Dawid AP, Smith AFM. eds Bayesian Satistics 5. Oxford: Oxford University Press; 1995: 599-607.
• 15 Diggle P, Heagerty P, Liang KY, Zeger S. Analysis of Longitudinal Data. Oxford: Oxford University Press; 2002