Diagnostic accuracy of artificial intelligence-enabled vectorcardiography versus myocardial perfusion SPECT in patients with suspected or known coronary heart disease

Abstract

Aim The present study evaluated with myocardial perfusion SPECT (MPS) the diagnostic accuracy of an artificial intelligence-enabled vectorcardiography system (Cardisiography, CSG) for detection of perfusion abnormalities.

Methods We studied 241 patients, 155 with suspected CAD and 86 with known CAD who were referred for MPS. The CSG was performed after the MPS acquisition. The CSG results (1) p-factor (perfusion, 0: normal, 1: mildly, 2: moderately, 3: highly abnormal) and (2) s-factor (structure, categories as p-factor) were compared with the MPS scores. The CSG system was not trained during the study.

Results Considering the p-factor alone, a specificity of >78% and a negative predictive value of mostly >90% for all MPS variables were found. The sensitivities ranged from 17 to 56%, the positive predictive values from 4 to 38%. Combining the p- and the s-factor, significantly higher specificity values of about 90% were reached. The s-factor showed a significant correlation (p=0.006) with the MPS ejection fraction.

Conclusions The CSG system is able to exclude relevant perfusion abnormalities in patients with suspected or known CAD with a specificity and a negative predictive value of about 90% combining the p- and the s-factor. Since it is a learning system there is potential for further improvement before routine use.

Keywords

chronic coronary syndrome, myocardial perfusion SPECT, vectorcardiography, artificial intelligence

Introduction

Noninvasive cardiac imaging is a main pillar in the diagnostic pathway of chronic coronary syndrome. Myocardial perfusion SPECT (MPS), stress-echocardiography and perfusion MRI as functional methods and cardiac CT as a morphological method are used. Exercise ECG plays a minor role due its low diagnostic accuracy and is recommended for the assessment of clinical aspects such as exercise tolerance, symptoms, arrhythmias, and blood pressure response. Resting ECG is one element in the initial diagnostic process of patients with chest pain [1] [2].

Recent developments found that modified vector analysis of the resting vectorcardiogram has the potential to detect ischaemia and thus to identify patients with coronary artery disease. The analysis evaluates the spatial and temporal heterogeneity of the cardiac excitation. Together with a supervised machine learning algorithm, this system is commercially available as Cardisiography (CSG, Cardisio GmbH, Frankfurt/Main, Germany). A detailed description has been published in a study that demonstrated a high diagnostic accuracy of CSG as compared with coronary angiography [3].

The present prospective study was designed to evaluate CSG with MPS in patients with suspected or known coronary artery disease (CAD). MPS served as the reference method.

Material and Methods

Patients

We studied 241 patients, 155 with suspected CAD and 86 with known CAD, who were referred for MPS from cardiologists or directly from our hospital. The patient characteristics are summarised in [Table 1]. All of them had an indication for MPS imaging, i.e. those with suspected CAD a medium clinical likelihood of 15 to 85% [2] [4].

Table 1 Patient characteristics.

*(4 patients with pathological ECG, three patients for follow-up, two known CAD-patients with suspected progression, one patient with non-specific sensation, one patient with MPS before non-cardiac surgery)
Male (%)	63.5%
Age	67 ± 10.7
Known CAD	86 (35.7%)
Suspected CAD	155 (64.3%)
Indication for MPS
Angina pectoris (typical or atypical)	37.7%
Dyspnoea (rest and/or exertion)	32.8%
Exclude CAD in patients with risk factors	17.8%
Asymptomatic	7.1%
Other*	4.6%
Cardiovascular risk factors
Arterial hypertension	72.2%
Dyslipidaemia	48.1%
Smoking history	35.7%
Diabetes mellitus	24.9%
Positive family history	34.9%
Obesity (BMI >30 kg/m²)	21.6%

All patients gave their written informed consent for the CSG. The study was approved by the local ethics committee of the Ruhr-University Bochum, Germany (Reg. No. 2020–593).

Myocardial perfusion SPECT

Myocardial perfusion SPECT was performed according to the current guidelines [4] [5]. The stress test was performed with ergometry or pharmacologically with adenosine or regadenoson in patients unable to exercise or not reaching their target heart rate with ergometry. Camera acquisition was performed with a SPECT-CT (Siemens Symbia Intevo with a 16-row CT; Siemens Healthineers, Germany) or with a Cardio MD camera (Philips Medical Systems, The Netherlands) with transmission sources. Thus, all MPS were reconstructed with attenuation correction. Furthermore, all scans were acquired as gated SPECT with 8 frames and analysed with AutoQUANT 7.2 (Cedars Sinai Medical Center, USA). In patients who performed ergometry, the acquisition started 30 min after the stress test and in those with pharmacological stress after 60 min.

For the analysis, SSS (summed stress score), SRS (summed rest score), and SDS (summed difference score) were generated on a standard 17-segment model based on a normal database. The scores were checked by an experienced nuclear medicine physician and in some cases adjusted to the visual polarmap without being aware of the CSG result. They were categorised as:

• score 0 to 3: normal,

• score 4 to 6: mildly abnormal,

• score 7 to 9: moderately abnormal, and

• score > 9: highly abnormal.

For the calculation of sensitivity, specificity, positive and negative predictive value the scores were dichotomised into ≤ 6 and > 6.

Furthermore, all scans were finally evaluated by at least one experienced nuclear medicine physician as:

• normal,

• mildly abnormal,

• moderately abnormal,

• highly abnormal.

Cardisiography

CSG is a vectorcardiography that uses artificial intelligence (supervised machine learning). The background is to teach the artificial intelligence to interpret and classify data sets correctly. The artificial intelligence of CSG is continuously trained with confirmed findings and thus improving in its accuracy. The CSG procedure and algorithm are described in detail elsewhere [3].

In contrast to electrocardiography, a three-dimensional recording of the electrical activity of the heart is carried out. The result is based on three factors: (1) CSG p-factor for myocardial perfusion, CSG s-factor for myocardial structure, and CSG a-factor for arrhythmia. The CSG a-factor (arrhythmia) was not considered in this study.

CSG was performed after the acquisition under the gamma camera to ensure a sufficiently long rest period after the stress test. The CSG was performed by H.K., experienced with the system. During the acquisition the patient was sitting on a chair. Five coloured electrodes were used. The red, green and yellow ones were fixed to the chest wall at equal distances from each other, the earth in the area of the right costal arch and the white electrode at the level of the heart in the back area. The measurement took four minutes. Thereafter, the data were sent to the central CSG computer in Frankfurt/Main (Germany).

The resulting report included a CSG evaluation with the three factors each scored as

• 0: normal,

• 1: mildly abnormal,

• 2: moderately abnormal, and

• 3: severely abnormal.

We compared two approaches of CSG parameters labelled single factor CSG and dual factor CSG.

In the single factor CSG only the p-factor was considered. A p-factor of 0 and 1 was regarded as normal and of 2 or 3 as pathological.

Dual factor CSG was a combination of the p-factor and the s-factor. A result with a p-factor of 0 or 1 independently of the s-factor was considered normal, as was a p-factor of 2 or 3 with an s-factor of 0 and 1. Only a p-factor of 2 or 3 and an s-factor of 2 and 3 were regarded as pathological.

For our analysis the state of CSG in March 2023, as it was for example used in ambulatory care practices at that time, was used. The system was not trained with the MPS results during the study.

Cardisio s-factor and ejection fraction

For analysis of the s-factor a comparison with the ejection fraction (EF) measured with MPS was performed. The s-factors 0–1 and 2–3 were each combined into one group. The EF threshold was set to 50%.

Statistical analysis

For the analysis we calculated with Excel the sensitivity, specificity, positive and negative predictive values using MPS as the gold standard. Pearson’s chi-square test was used to compare sensitivity, specificity, negative and positive predictive value of the two CSG approaches, to compare groups with discrepancies in their MPS and CSG results, and to check the concordance between MPS ejection fraction and CSG s-factor. A p-value of <0.05 was considered statistically significant.

Results

Single factor CSG vs MPS

The CSG results of the single p-factor compared with the MPS parameters SSS (ischaemia and scar), SRS (scar), SDS (ischaemia), and the final conclusion are compiled in [Table 2].

Table 2 Single p-factor CSG vs MPS.

	p-factor vs.	Sensitivity	Specificity	Positive predictive value	Negative predictive value
Suspected CAD	SSS	30.0%	83.5%	11.1%	94.5%
	SRS	33.3%	82.9%	3.7%	98.4%
	SDS	16.7%	82.6%	3.7%	96.1%
	Conclusion	22.2%	82.9%	7.4%	94.5%
Known CAD	SSS	34.8%	79.4%	38.1%	76.9%
	SRS	35.3%	78.3%	28.6%	83.1%
	SDS	50.0%	77.5%	14.3%	95.4%
	Conclusion	55.6%	79.2%	23.8%	93.9%
Total	SSS	33.3%	82.2%	22.9%	88.6%
	SRS	35.0%	81.5%	14.6%	93.3%
	SDS	33.3%	80.8%	8.3%	95.9%
	Conclusion	38.9%	81.6%	14.6%	94.3%

There was an overall specificity of >78% and a negative predictive value of mostly >90% for most of the MPS variables irrespective of the state of disease. The sensitivities ranged from 17 to 56%, the positive predictive values from 4 to 38%. Sensitivities and positive predictive values tended to be higher in patients with known CAD, specificities and negative predictive values lower.

Match of single factor CSG and MPS conclusion

[Table 3] shows the matches between the CSG p-factor and the MPS conclusion over the defined categories.

Table 3 Concordance of CSG p-factor and MPS conclusion.

	MPS conclusion
CSG (p-factor)	normal	mildly abn.	mod. abn.	highly abn.	Sum
1complete match between CSG and MPS. 2discrepancy of one range. 3discrepancy of two ranges. 4discrepancy of three ranges.
0 (normal)	94 (39%)1	11 (5%)2	2 (1%)3	3 (1%)4	110 (46%)
1 (mildly abn.)	65 (27%)2	11 (5%)1	6 (2%)2	1 (0.4%)3	83 (34%)
2 (mod. abn.)	36 (15%)3	5 (2%)2	6 (2%)1	1 (0.4%)2	48 (20%)
3 (severely abn.)	0 (0%)4	0 (0%)3	0 (0%)2	0 (0%)1	0 (0%)
Sum	195 (81%)	27 (12%)	14 (5%)	5 (2%)	241 (100%)

An exact match was found in 111 patients (46%) of whom 94 (85%) were assessed as normal by both the CSG p-factor and the MPS conclusion.

A discrepancy of one range between MPS and CSG was found in 88 patients (37%), of two ranges in 39 (16%), and of three in 3 patients (1.2%). A patient example of the last of these is shown in [Fig. 1].

Fig. 1 Patient with large ischaemia (SDS = 18) with normal single factor CSG and also with normal dual factor CSG.Angiography revealed a proximal LAD stenosis and a high grade RCA stenosis. Subsequently, the patient underwent bypass surgery.Ex: Exercise, Re: Rest.

Of the 222 patients with a normal or mildly abnormal MPS conclusion, 41 (17%) had a pathological CSG result. In 12 cases (5%) the constellation was vice versa.

In both of these groups no significant difference was found in cardiovascular risk factors (hypertension, dyslipidaemia, diabetes mellitus) compared to the respective group with concordant CSG and MPS conclusion. The same applied to gender, atrial fibrillation and left bundle branch block.

Dual factor CSG vs MPS

The results of this approach, compared to the MPS parameters are listed in [Table 4]. As in [Table 2], sensitivities and positive predictive values were low and specificities and negative predictive values high for all MPS parameters. Compared to the single p-factor CSG the combination shifted specificity values for SSS and MPS conclusion significantly from about 80% to > 90% in all categories. The negative predictive valued remained nearly unchanged. The positive predictive value increased by about 5 to 7% points. The changes were insignificant.

Table 4 Dual factor CSG vs MPS.

	p-factor + s-factor vs.	Sensitivity	Specificity	Positive predictive value	Negative predictive value
In brackets difference to single p-factor CSG vs MPS from [Table 2]. *p<0.05 vs single p-factor CSG
Suspected CAD	SSS	30.0% (0%)	93.8%* (+10.3)	25.0% (+13.9%)	95.1% (+0.6%)
	SRS	33.3% (0%)	92.8% (+9.9%)	8.3% (+4.6%)	98.6% (+0.2%)
	SDS	16.7% (0%)	92.6% (+10.0%)	8.3% (+4,6%)	96.5% (+0.4%)
	Conclusion	22.2% (0%)	93.2% (+10.3%)	16.7% (+9,3%)	95.1% (+0.6%)
Known CAD	SSS	13.0% (–21.8%)	93.7%* (+14,3%)	42.9% (+4.8%)	74.7% (–2.2%)
	SRS	17.7% (–17.6%)	94.2% (+15.9%)	42.9% (+14.3%)	82.3% (–0.8%)
	SDS	0%* (–50.0%)	91.3% (+13.8%)	0% (–14.6%)	92.4% (–1.1%)
	Conclusion	22.2% (–33.4%)	93.5% (+14.3%)	28.6% (+4.8%)	91.1% (–2.8%)
Total	SSS	18.2% (–15.1%)	93.8%* (+11.6%)	31.6% (+8.7%)	87.8% (+1.1%)
	SRS	20.0% (–20.0%)	93.2% (+11.7%)	21.1% (+6.5%)	92.8% (–0.5%)
	SDS	8.3% (–15.0%)	92.1% (+11.3%)	5.3% (+3.0%)	95.1% (–0.8%)
	Conclusion	22.2% (–16.7%)	93.3%* (+11.7%)	21.1% (+6.5%)	93.7% (–0.6%)

Sensitivities and negative predictive values tended to be higher in patients with suspicion of CAD, specificities to be on the same level, and positive predictive values lower.

CSG s-factor and MPS ejection fraction

The results of the s-factor (structural factor) vs MPS EF are given in [Table 5]. For the total group the sensitivity was 26%, specificity 89%, PPV 38%, and NPV 82%. S-factor and EF showed a significant concordance (p=0.006).

Table 5 CSG s-factor vs EF.

CSG (s-factor)	MPS	MPS	Sum
	EF > 50%	EF < 50%
0–1	170 (71%)	21 (9%)	191 (80%)
2–3	37 (15%)	13 (5%)	50 (20%)
total	207 (86%)	34 (14%)	241 (100%)

Discussion

The present study evaluated with MPS a vectorcardiography artificial intelligence-based system for non-invasive detection of myocardial perfusion abnormalities/ischaemia without stress testing. MPS served as the reference method. There is still limited experience with CSG based on studies. So far, only one other study on this method has been published (as of 12/2023) [3].

The key finding of the present study is that CSG is able to exclude significant perfusion disturbances in terms of ischemia with specificities and negative predictive values > 90%. The accuracy to exclude myocardial ischaemia can be improved if the CSG p-factor (perfusion) is not considered alone, but combined with the CSG s-factor (structure).

On the other hand, sensitivity and positive predictive values with both approaches are low, which indicates that CSG is too sensitive. This is a frequent finding of systems based on artificial intelligence [6]. A positive CSG result is therefore not necessarily associated with significant amounts of ischaemia in the left ventricular myocardium.

The CSG s-factor alone shows a similar diagnostic performance with low sensitivity and low positive predictive value, but high specificity and high negative predictive value for significant myocardial damage with a reduced EF in MPS. The concordance of the CSG s-factor and the EF is probably due to the close relationship between structural damage and reduced EF [7].

With both approaches (CSG p-factor alone or combination of CSG p- and s-factor) we found a small proportion of patients with discrepant CSG and MPS results. Based on the available clinical risk factors, arrhythmia, and conduction disturbances, a key feature compared to the respective group with matching results to explain the discrepancies could not be found. The reasons may lie in other factors which we did not cover in this study or may be eliminated or reduced in the further learning process of the system.

In the only comparative study published to date, Braun et al found a sensitivity of > 90% and a specificity of > 70% of CSG vs CAD detected with coronary angiography [3]. In particular, the sensitivity was considerably higher than in the present study. In the study of Braun et al, however, the CAD definition was taken very broad and patients were CAD positive if a 1, 2 or 3-vessel disease was detected, regardless of the size and the location of the stenosis. In contrast to our study, ischaemia was not taken into account.

Another issue to consider in this context is that patients who have invasive coronary angiography are likely to have pathological findings which result in a referral bias and may have masked the hypersensitivity of CSG.

Furthermore, in the study of Braun et al, the artificial intelligence of the CSG was trained with the dataset of the patients, with a proportion of 80% used for training and 20% for testing.

In the present study there was no training phase and the artificial intelligence of the CSG remained untrained for our data. CSG was tested using the particular state of March 2023.

All these aspects may explain why our data differ from those of Braun et al., but seem to be nearer to an application of CSG in routine daily practice [3].

In Germany in 2024, CT cardiac angiography will be an element of statutory health care and thus increase the numbers of noninvasive angiographies. In this context the performance of CSG will be of interest as compared to the results of Braun et al. [3].

A diagnostic pathway is conceivable which excludes patients with CSG negative results from further diagnosis and shifts positives to non-invasive imaging. Our data indicate that about 10% of the normal CSG patients would then be falsely classified as not having functionally relevant CAD. Thus, further evaluation of CSG after continuing training and adjustment is advisable before any recommendation of routine use in daily practice.

It should be considered that, due to the indication for MPS imaging, CSG was only tested in a cohort with a medium clinical likelihood of 15 to 85% [4]. In daily routine of primary care, pretest probabilities or clinical likelihoods are often < 15%, and the CSG performance may improve under such conditions. Its role can be compared with the exercise ECG which has a low sensitivity but a high specificity and its field of application in low pretest ranges [8].

The concept of diagnosing perfusion abnormalities which usually occur at stress in a resting condition is initially surprising but also promising. We are about to test the diagnostic performance of CSG during the MPS exercise or pharmacological stress test and expect to achieve further improvements and insights.

Conclusion

The CSG system is able to exclude relevant perfusion abnormalities in patients with suspected or known CAD with a specificity and a negative predictive value of about 90% combining the p- and the s-factor. Since it is a learning system there is potential for further improvement before routine use. Thereafter, such systems may help to select patients more precisely for MPS and other non-invasive imaging methods.

Conflict of Interest

Mr. Knobl who performed the CSG measurements receives a salary from the company Cardisio GmbH as a freelancer. The other authors declare that they have no conflicts of interest.

• References

• 1 Bundesärztekammer AdwmF, Kassenärztliche Bundesvereinigung. Nationale Versorgungs-Leitlinie Chronische KHK (Langfassung), 6. Auflage. AWMF-Register-Nr.: nvl-004. 2022 www.leitlinien.de/nvl/khk
• 2 Knuuti J, Wijns W, Saraste A. et al. 2019 ESC Guidelines for the diagnosis and management of chronic coronary syndromes. Eur Heart J 2020; 41: 407-477 DOI: 10.1093/eurheartj/ehz425. (PMID: 31504439)
• 3 Braun T, Spiliopoulos S, Veltman C. et al. Detection of myocardial ischemia due to clinically asymptomatic coronary artery stenosis at rest using supervised artificial intelligence-enabled vectorcardiography – A five-fold cross validation of accuracy. J Electrocardiol 2020; 59: 100-105 DOI: 10.1016/j.jelectrocard.2019.12.018. (PMID: 32036110)
• 4 Nuklearmedizin DGf. S1-Leitlinie Myokard-Perfusions-SPECT(-CT). AWMF Registernummer 031 – 006. 2023 https://register.awmf.org/de/leitlinien/detail/031–006
• 5 Verberne HJ, Acampa W, Anagnostopoulos C. et al. EANM procedural guidelines for radionuclide myocardial perfusion imaging with SPECT and SPECT/CT: 2015 revision. Eur J Nucl Med Mol Imaging 2015; 42: 1929-1940
• 6 Topol EJ. High-performance medicine: the convergence of human and artificial intelligence. Nature Medicine 2019; 25: 44-56 DOI: 10.1038/s41591-018-0300-7. (PMID: 30617339)
• 7 McDonagh TA, Metra M, Adamo M. et al. 2021 ESC Guidelines for the diagnosis and treatment of acute and chronic heart failure: Developed by the Task Force for the diagnosis and treatment of acute and chronic heart failure of the European Society of Cardiology (ESC) With the special contribution of the Heart Failure Association (HFA) of the ESC. European Heart Journal 2021; 42: 3599-3726
• 8 Albus C, Barkhausen J, Fleck E. et al. Diagnostik der chronischen KHK. Dtsch Arztebl Int 2017; 114: 712-719

Publication History

Received: 20 September 2023

Accepted after revision: 08 January 2024

Article published online:
20 February 2024