Methods Inf Med 1990; 29(04): 308-316 DOI: 10.1055/s-0038-1634794
ECG Interpretation Systems
Schattauer GmbH
Evaluation of ECG Interpretation Results Obtained by Computer and Cardiologists[*]
J. L. Willems
1
CSE Coordinating Center, Univ. of Leuven, Belgium
,
C. Abreu-Lima
2
Department of Cardiology, University of Porto, Portugal
,
P. Arnaud
3
INSEPvM, Unité 121, Hôp. Cardiovasculaire, Lyon, France
,
C. R. Brohet
4
Department of Cardiology, University of Louvain, Brussels, Belgium
,
B. Denis
5
Service de Cardiologie, Univ. of Grenoble, France
,
J. Gehring
6
Clinic for Cardiovascular Diseases, Höhenried, Federal Republic of Germany
,
I. Graham
7
Adelaide Hospital, Dublin, Ireland
,
G. van Herpen
8
University Hospital, Leiden, The Netherlands
,
H. Machado
9
University Hospital Santa Maria, Lisbon, Portugal
,
J. Michaelis
10
Institut für Medizinische Statistik und Dokumentation, Mainz, Federal Republic of Germany
,
S. D. Moulopoulos
11
Department of Therapeutics, University of Athens, Athens, Greece
› InstitutsangabenThe research presented in this paper has been supported by the European Commission within the frame of its 2nd, 3rd and 4th Medical and Public Health Research Programmes of DG XII, Section Biomedical Engineering, and by various funding agencies in ten Member States of the European Community. The authors are grateful to Ms. I. Tassens for secretarial assistance and to Mr. I. Schoolmeesters for computer programming help.
In an international project investigators from 25 institutes are trying to establish a common reference library and evaluation methods for testing the diagnostic performance of various ECG computer programs and of cardiologists, based on ECG-independent clinical information. A first set of 500 validated ECGs was collected and analyzed by fifteen different computer programs and nine cardiologists, seven of who analysed the ECG and five the VCG. A coding scheme was used to map individual diagnostic statements onto a common set. Combined program and referee results were obtained by weighted averaging. Preliminary results indicate that the classification accuracy of several programs can still be improved. However, it was also apparent that the results of the best 12-lead ECG computer programs proved to be almost as accurate as the best of seven cardiologists in classifying seven main disease categories, i.e., normal, left, right and biventricular hypertrophy, anterior, inferior and combined myocardial infarction. Evaluation of rhythm statements and conduction disturbances was not included in the study. The data collection is still being pursued in order to reach over 1,000 cases. In this way a common diagnostic database is being established for comparative testing of diagnostic computer programs. This should lead to consumer protection and improve the accuracy and reliability of computerized electrocardiography.
1
Bruce RA,
Yarnall SR.
Reliability and normal variations of computer analysis of Frank electrocardiograms by the Smith-Hyde program. Am J Cardiol 1968; 29: 389-96.
2
Yokoi M,
Yasui S,
Okajima M,
Mizuno Y.
Comparative studies on the diagnostic accuracy between the two diagnostic logic in electrocardiographic interpretation. Jpn Circ J 1969; 33: 51-8.
4
Bailey JJ,
Itscoitz SB,
Hirshfield Jr JW,
Grauer LE,
Horton MB.
A method for evaluating computer programs for electrocardiographic interpretation. I. Application to the experimental IBM program of 1971. Circulation 1974; 50: 73-9.
5
Pipberger HV,
Cornfield J.
What ECG-computer program to choose for clinical application. The need for consumer protection. Circulation 1973; 47: 918-20.
6
Meyer J,
Heinrich KW,
Merx W,
Efferts S.
Computeranalyse des Elektrocardiogramms mit verschiedenen Programmen. I. Formanalyse. Dtsch Med Wochenschr 1974; 23: 1213-23.
7
Smets Ph,
Willems JL,
Talmon J,
De Maertelaer V,
Kornreich F.
Methodology for the comparison of various diagnostic procedures. Biometrie-Praximetrie 1975; 15: 3-35.
11
Feruglio GA,
Feraco E,
Mirolo R.
Experience of a user of an ECG interpretation system. In:
van Bemmel JH,
Willems JL.
eds. Trends in Computer-Processed Electrocardiograms. Amsterdam: North-Holland, Publ Comp; 1977: 239-47.
12
Rautaharju PM,
Ariet M,
Pryor TA.
et al. The quest for optimal electrocardiography. Task Force III on computers in diagnostic electrocardiography. Am J Cardiol 1978; 41: 155-71.
13
Bourdillon PJ,
Kilpatrick D.
Clinicians, the Mount Sinai program and the Veterans Administration program evaluated against clinico-pathological data derived independently of the electrocardiogram. Eur J Cardiol 1978; 08: 395-412.
15
Bonner R,
Jackson LK,
Hooper JK,
Smith CH,
Hsieh RC.
A method for testing performance in electrocardiographic interpretation. In:
Pryor AT,
Bailey JJ.
eds. Computerized Interpretation of the ECG, TV. New York: Engineering Foundation; 1979: 126-44.
16
Brohet C,
Richman HG.
Clinical evaluation of automated processing of electrocardiograms by the Veterans Administration Program (AVA 3.4). Am J Cardiol 1979; 43: 1167-74.
17
Bailey JJ,
Harris EK.
Evaluation of ECG interpretation: truth versus beauty. In:
Tolan GD,
Pryor TA.
eds. Computerized Interpretation of the Electrocardiogram, V. New York: Engineering Foundation; 1980: 179-82.
19
Laks MM.
“Gold standard” for ECG diagnosis - revisited 1981. In:
Bonner RE,
Pryor TA,
Laks MM,
Cole SS.
eds. Computerized Interpretation of the ECG, VI. New York: Engineering Foundation; 1982: 267-75.
20
Bailey JJ,
Berson A,
Jackson L,
Milliken J,
Stevens J,
Tolan G,
Wolf H.
Evaluation methodologies for ECG diagnostic systems. In:
Bonner RE,
Pryor TA,
Laks MM,
Colle SS.
eds. Computerized Interpretation of the ECG, VI. New York: Engineering Foundation; 1981: 53-62.
21
Wolf HK,
Milliken JA,
Gregor RD,
Burg-graf G,
MacCahn J,
Rautaharju PM,
Cuddy TE.
Accuracy of type “A” statements of human and computer ECG classifiers. In:
Laks MM,
Colle SS.
eds. Computerized Interpretation of the Electrocardiogram, VII. New York: Engineering Foundation; 1982: 335-40.
22
Bernard P,
Chairman BR,
Scholl JM,
Val PG,
Chabot M.
Comparative diagnostic performance of the Telemed computer program. J Electrocardiol 1983; 16: 97-102.
23
Graham IM,
Reardon B,
Mulcahy D,
Mulcahy R.
User evaluation of a commercially available computerized electrocardiogram interpretation system. In:
Willems JL,
van Bemmel JH,
Zywietz C.
eds. Computer ECG Analysis: Towards Standardization. Amsterdam: North-Holland, Publ Comp; 1986: 191-3.
26
Pipberger HV,
McCaughan D,
Littmann D.
et al. Clinical application of a second generation electrocardiographic computer program. Am J Cardiol 1975; 35: 597-608.
27
Harris LK,
Blackburn HW,
Prineas RJ,
Wolf HK.
Electrocardiographic diagnosis of myocardial infarction by human readers and two computer programs. In: Abstract Book I. European Congress on Cardiology; Paris: 1976: 725.
29
Kornreich F,
Block P,
Bourgain R.
et al. Multigroup diagnosis classification with a new “maximal” lead system. In:
Abel H.
ed. Advances in Cardiology and Electro-cardiology. Basel: S. Karger; 1976: 121-7.
30
Khadr NE,
Bray CL,
Beton DC,
Croxson RS,
Hughes M,
Jeffrey C,
Richards B.
Diagnosis of left ventricular hypertrophy and myocardial infarction by Bonner/IBM Program verified by ECG-independent evidence. In:
Ripley KL,
Ostrow HG.
eds. Computers in Cardiology. Long Beach: IEEE Computer Society; 1979: 93-7.
32
Willems JL,
Lesaffre E,
Pardaens J,
De Schreye D.
Multigroup logistic classification of the 12- and 3-lead ECG. In:
Willems JL,
van Bemmel JH,
Zywietz C.
eds. Computer ECG Analysis: Towards Standardisation. Amsterdam: North-Holland Publ Comp; 1986: 203-10.
33
Willems JL,
Lesaffre E,
Pardaens J.
Comparison of the classification ability of the electrocardiogram and vectorcardiogram. Am J Cardiol 1987; 59: 119-24.
34
Willems JL,
Arnaud P,
van Bemmel JH,
Degani R,
Macfarlane PW,
Zywietz C.
Common standards for quantitative electrocardiography: goals and main results. Meth Inform Med 1990; 29: 263-71.
39
Macfarlane PW.
A hybrid lead system for routine electrocardiography. In:
Macfarlane PW.
ed. Progress in Electrocardiography. London: Pittman Medical Publ; 1979: 1-5.
41
Diamond GA,
Hirsch M,
Forrester JS.
et al. Application of information theory to clinical diagnostic testing. The electrocardiographic stress test. Circulation 1981; 63: 915-21.
47
Simonson E,
Tuna N,
Okamoto N.
et al. Diagnostic accuracy of the electrocardiogram and vectorcardiogram. A cooperative study. Am J Cardiol 1966; 17: 829-78.
50
Rautaharju PM,
Blackburn HW,
Wolf HK,
Horacek M.
Computers in clinical electrocardiology. Is vectorcardiography becoming obsolete?. In:
Abel H.
ed. Electrocardiology. Advances in Cardiology. Basel: Karger; 1976: 143-56.
51
Dudeck J,
Michaelis J.
Problems in the diagnostic process in electrocardiography. In:
Zywietz Chr,
Schneider B.
eds. Computer Application on ECG and VCG Analysis. Amsterdam: North-Holland Publ Comp; 1973: 283-95.
