Download PDF
Methods Inf Med 2006; 45(06): 622-630
DOI: 10.1055/s-0038-1634126
Editorial Note
Schattauer GmbH

Use of Statistical Techniques to Synthesize Explicit Criteria Developed by an Expert Panel

I. Arostegui
1   Departamento de Matemática Aplicada, Estadística e Investigación Operativa, Universidad del País Vasco, Leioa, Bizkaia, Spain
,
J. M. Quintana
2   Unidad de Investigación, Hospital de Galdakao, Galdakao, Bizkaia, Spain
,
A. Urkaregi
1   Departamento de Matemática Aplicada, Estadística e Investigación Operativa, Universidad del País Vasco, Leioa, Bizkaia, Spain
› Author Affiliations
Further Information

Publication History

Received 04 October 2005

accepted 08 March 2006

Publication Date:
08 February 2018 (online)

    Permissions and Reprints

Summary

Objectives: Methodology based on expert panels has been commonly used to evaluate the appropriateness of interventions. An important issue is the adequate synthesis of the generated information in an applicable way to clinical decision making. This paper shows how statistical procedures help synthesize the results of an expert panel.

Methods: Three statistical techniques were applied to an expert panel that developed explicit criteria to assess the appropriateness of total hip joint replacement: classification tree, regression tree and multiple correspondence analysis combined with automatic classification.

Results: Results provided by the three models were shown in graphical displays and were compared to the original panel results using crude and weighted probability of misclassification. Results were also applied to real interventions in order to know the implication of the misclassification on real patients.

Conclusions: The statistical techniques help summarize data from panels of experts and provide useful decision models for clinical practice, especially when the number of indications is big. However, degree of misclassification and its implication should be taken into account.

Keywords

Appropriateness - statistical models - classification error - clinical guidelines
 

  • References

  • 1 Wennberg J. The paradox of appropriate care. JAMA 1987; 258: 2568-9.
    CrossrefPubMedGoogle Scholar
  • 2 Hadorn DC, Brook RH. The health care resource allocation debate. Defining our terms. JAMA 1991; 266: 3328-31.
    CrossrefPubMedGoogle Scholar
  • 3 Brook RH, Chassin MR, Fink A, Solomon DH, Kosecoff J, Park RE. A method for the detailed assessment of the appropriateness of medical technologies. Int J Technol Assess Health Care 1986; 2: 53-63.
    CrossrefPubMedGoogle Scholar
  • 4 Park RE, Fink A, Brook RH, Chassin MR, Kahn KL, Merrick NJ, Kosecoff J, Solomon DH. Physician ratings of appropriate indications for six medical and surgical procedures. Am J Public Health 1986; 76: 766-72.
    CrossrefPubMedGoogle Scholar
  • 5 Gale RP, Park RE, Dubois RW, Herzing GP, Hocking WG, Horowitz MM, Keating A, Kenpin S, Linker CA, Schiffer CA, Wiernik PH, Weisdorf DJ, Rai KR. Delphi-panel analysis of appropriateness of high-dose therapy and bone marrow transplants in chronic myelogenous leukemia in chronic phase. Leuk Res 1999; 23: 817-26.
    CrossrefPubMedGoogle Scholar
  • 6 Merrick NJ, Fink A, Park RE, Brook RH, Kosecoff J, Chassin MR, Solomon DH. Derivation of clinical indications for carotid endarterectomy. Am J Public Health 1987; 77: 187-90.
    CrossrefPubMedGoogle Scholar
  • 7 Wietlisbach V, Vader JP, Porchet F, Constanza MC, Burnand B. Statistical approaches in the development of clinical practice guidelines from expert panels. Med Care 1999; 37: 785-97.
    CrossrefPubMedGoogle Scholar
  • 8 Intrator O, Kooperberg C. Trees and splines in survival analysis. Stat Methods Med Res 1995; 4: 237-61.
    CrossrefPubMedGoogle Scholar
  • 9 Landrum MT, Normand SLT. Applying Bayesian ideas for the development of medical guidelines. Stat Med 1999; 18: 117-37.
    CrossrefPubMedGoogle Scholar
  • 10 Uebersax JS. Statistical modeling of expert ratings in medical treatment appropriateness. J Am Stat Assoc 1993; 88: 421-27.
    CrossrefPubMedGoogle Scholar
  • 11 Kattan MW, Hess KR, Beck JR. Experiments to determine whether Recursive Partitioning (CART) or an artificial neural network overcomes theoretical limitations of Cox proportional hazards regression. Comput Biomed Res. 1998; 31: 363-73.
    CrossrefPubMedGoogle Scholar
  • 12 Speroff T, Connors AF, Dawson NV. Lens model analysis of hemodynamic status in the critically ill. Med Decis Making 1989; 9: 243-52.
    CrossrefPubMedGoogle Scholar
  • 13 Benzécri JP. Statistical analysis as a tool to make patterns emerge from data. In Watanabe S. (ed.) Methodologies of Pattern Recognition. New York: Academic Press; 1969. pp 35-74.
    Google Scholar
  • 14 Quintana JM, Arostegui I, Azkarate J, Goenaga JI, Elexpe X, Letona J, Arcelai A. Development of explicit criteria for total hip joint replacement. J Clin Epidemiol. 2000; 53: 1200-8.
    CrossrefPubMedGoogle Scholar
  • 15 Zhang H, Singer B. Recursive Partitioning in the Health Sciences. New York: Springer-Verlag; 1999
    Google Scholar
  • 16 Segal MR. Extending the elements of tree-structured regression. Stat Methods Med Res 1995; 4: 219-36.
    CrossrefPubMedGoogle Scholar
  • 17 Venables WN, Ripley BD. Modern Applied Statistics with S-Plus. New York: Springer-Verlag; 1994
    Google Scholar
  • 18 Greenacre M. Correspondence analysis in medical research. Stat Methods Med Res 1992; 1: 97-117.
    CrossrefPubMedGoogle Scholar
  • 19 Jambu M. Classification Automatique pour l’Analyse des Données. Tome 1: Méthodes, et algoritmes. Paris: Dunod; 1978
    Google Scholar
  • 20 Ward JH. Hierarchical grouping to optimize an objective function. J Am Stat Assoc 1963; 58
    PubMedGoogle Scholar
  • 21 SAS Institute Inc. SAS Procedures Guide, Version 6. Cary, NC: SAS Institute 1994
    PubMedGoogle Scholar
  • 22 S-PLUS 4. Guide to Statistics. Data Analysis Products division. Seattle, WA: MathSoft, Inc.; 1997
    Google Scholar
  • 23 Lébart L, Morineau A, Lambert T. SPAD. N: Systême Portable pour l’Analyse des Données. Manuel de réference. Sèvres: CISIA 1987
    PubMedGoogle Scholar
  • 24 Faulkner A, Kennedy LG, Baxter K, Donovan J, Wilkinson M, Bevan G. Effectiveness of hip prostheses in primary total hip replacement: a critical review of evidence and economic model. Health Technol Assessment 1998; 2: 1-133.
    PubMedGoogle Scholar
  • 25 Shekelle PG, Shriger DL. Evaluating the use of the appropriateness method in the Agency for Health Care Policy and Research Clinical Practice Guideline development process. Health Serv Res 1996; 31: 453-68.
    PubMedGoogle Scholar
  • 26 Shekelle PG, Kahan JP, Bernstein SJ, Leape LL, Kamberg CJ, Park RE. The reproducibility of a method to identify the overuse and underuse of medical procedures. N Engl J Med 1998; 338: 1888-95.
    CrossrefPubMedGoogle Scholar
  • 27 Leape LL, Park RE, Kahan JP, Brook RH. Group judgments of appropriateness: the effect of panel composition. Qual Assur Health Care 1992; 4: 151-9.
    PubMedGoogle Scholar
  • 28 Bernstein SJ, Kosecoff J, Gray D, Hampton JR, Brook RH. The appropriateness of the use of cardiovascular procedures. British versus U.S. perspectives. Int J Technol Assess Health Care 1993; 9: 3-10.
    CrossrefPubMedGoogle Scholar
  • 29 Kahan JP, Park RE, Leape LL, Bernstein SJ, Hilborne LH, Parker L, Kamberg CJ, Ballard DJ, Brook RH. Variations by specialty in physician ratings of the appropriateness and necessity of indications for procedures. Med Care 1996; 34: 512-23.
    CrossrefPubMedGoogle Scholar
  • 30 Phelps CE. The methodologic foundations of studies of the appropriateness of medical care. N Engl J Med 1993; 329: 1241-5.
    CrossrefPubMedGoogle Scholar
  • 31 Hicks NR. Some observations on attempts to measure appropriateness of care. BMJ 1994; 309: 730-3.
    CrossrefPubMedGoogle Scholar
  • 32 Phelps CE. Appropriateness studies. N Engl J Med 1994; 330: 433-4.
    PubMedGoogle Scholar
  • 33 Schilling J, Faisst K, Kapetanios E, Wyss P, Norrie MC, Gutzwiller F. Appropriateness and necessity research on the internet: Using a “second opinion system”. Methods Inf Med 2000; 39: 233-7.
    Article in Thieme ConnectPubMedGoogle Scholar
  • 34 Naylor CD, Williams J. Primary hip and knee replacement surgery: Ontario criteria for case selection and surgical priority. Qual Health Care 1996; 5: 20-30.
    CrossrefPubMedGoogle Scholar
  • 35 McConnochie KM, Roghmann KJ, Pasternack J. Developing prediction rules and evaluating observation patterns using categorical clinical markers: two complementary procedures. Med Decis Making 1993; 13: 30-42.
    CrossrefPubMedGoogle Scholar
  • 36 Marshall RJ. The use of classification and regression trees in clinical epidemiology. J Clin Epidemiol 2001; 54: 603-9.
    CrossrefPubMedGoogle Scholar
  • 37 Leclerc A, Luce D, Lert F, Chastang JF, Logeay P. Correspondence analysis and logistic modeling: complementary use in the analysis of a health survey among nurses. Stat Med 1988; 7: 983-95.
    CrossrefPubMedGoogle Scholar
  • 38 Escofier B, Pagès J. Analyses factorielles simples et multiples. Paris: Dunod 1990
    PubMedGoogle Scholar
  • 39 Maurer W. Creative and innovative statistics in clinical research and development. Methods Inf Med 2005; 44: 551-60.
    Article in Thieme ConnectPubMedGoogle Scholar