Fully Automated Artery-Specific Calcium Scoring Based on Machine Learning in Low-Dose Computed Tomography Screening

Abstract

Purpose Evaluation of machine learning-based fully automated artery-specific coronary artery calcium (CAC) scoring software, using semi-automated software as a reference.

Methods A total of 505 patients underwent non-contrast-enhanced calcium scoring computed tomography (CSCT). Automated, machine learning-based software quantified the Agatston score (AS), volume score (VS), and mass score (MS) of each coronary artery [right coronary artery (RCA), left main (LM), circumflex (CX) and left anterior descending (LAD)]. Identified CAC of readers who annotated the data with semi-automated software served as a reference standard. Statistics included comparisons of evaluation time, agreement of identified CAC, and comparisons of the AS, VS, and MS of the reference standard and the fully automated algorithm.

Results The machine learning-based software correlated strongly with the reference standard for the AS, VS, and MS (Spearmanʼs rho > 0.969) (p < 0.001), with excellent agreement (ICC > 0.919) (p < 0.001). The mean assessment time of the reference standard was 59 seconds (IQR 39–140) and that of the automated algorithm was 5.9 seconds (IQR 3.9–16) (p < 0.001). The Bland-Altman plots mean difference and 1.96 upper and lower limits of agreement for all arteries combined were: AS 0.996 (1.33 to 0.74), VS 0.995 (1.40 to 0.71), and MS 0.995 (1.35 to 0.74). The mean bias was minimal: 0.964–1.0429. Risk class assignment showed high accuracy for the AS in total (weighed κ = 0.99) and for each individual artery (κ = 0.96–0.99) with corresponding correct risk group assignment in 497 of 505 patients (98.4 %).

Conclusion The fully automated artery-specific coronary calcium scoring algorithm is a time-saving procedure and shows excellent correlation and agreement compared with the clinically established semi-automated approach.

Key points: 

• Very high correlation and agreement between fully automatic and semi-automatic calcium scoring software.

• Less time-consuming than conventional semi-automatic methods.

• Excellent tool for artery-specific calcium scoring in a clinical setting.

Citation Format

• Winkelmann MT, Jacoby J, Schwemmer C et al. Fully Automated Artery-Specific Calcium Scoring Based on Machine Learning in Low-Dose Computed Tomography Screening. Fortschr Röntgenstr 2022; 194: 763 – 770

Zusammenfassung

Ziel Evaluierung einer auf maschinellem Lernen basierenden vollautomatischen arterienspezifischen Software zur Bewertung des Koronarkalkes (CAC), unter Verwendung einer halbautomatischen Software als Referenz.

Methoden Bei insgesamt 505 Patienten wurde eine nicht kontrastverstärkte Calcium-Scoring-Computertomografie (CSCT) durchgeführt. Eine automatisierte, auf Machine Learning basierende Software quantifizierte den Agatston-Score (AS), Volumen-Score (VS) und Massen-Score (MS) jeder Koronararterie (rechte Koronararterie [RCA], linke Koronararterie [LM], Ramus circumflexus [CX] und Ramus interventricularis anterior [LAD]). Ermittelte CAC der Reader, die die Daten mit einer halbautomatischen Software annotierten, dienten als Referenzstandard. Die Statistik umfasste Vergleiche der Auswertungszeit, Übereinstimmung der identifizierten CAC sowie Vergleiche von AS, VS und MS des Referenzstandards und vollautomatischen Algorithmus.

Ergebnisse Die auf maschinellem Lernen basierende Software korrelierte stark mit dem Referenzstandard für AS, VS und MS (Spearmanʼs rho > 0,969) (p < 0,001), mit hervorragender Übereinstimmung (ICC > 0,919) (p < 0,001). Die mittlere Bewertungszeit des Referenzstandards betrug 59 s (IQR 39–140) und die des automatischen Algorithmus 5,9 s (IQR 3,9–16) (p < 0,001). Die mittlere Differenz der Bland-Altman-Plots und die bei 1.96 × Standardabweichung definierten oberen und unteren Grenzen der Übereinstimmung für alle Arterien zusammen betrugen: AS 0,996 (1,33 bis 0,74), VS 0,995 (1,40 bis 0,71), und MS 0,995 (1,35 bis 0,74). Der mittlere Bias war minimal: 0,964–1,0429. Die Risikoklassenzuordnung zeigte eine hohe Genauigkeit für den AS in Summe (gewichtetes κ = 0,99) und für jede Arterie (κ = 0,96–0,99) mit entsprechender korrekter Risikogruppenzuordnung bei 497 von 505 Patienten (98,4 %).

Schlussfolgerung Der vollautomatische arterienspezifische Koronarkalk-Scoring-Algorithmus ist ein zeitsparendes Verfahren und zeigt eine hervorragende Korrelation und Übereinstimmung mit dem klinisch etablierten halbautomatischen Ansatz.

Kernaussagen: 

• Sehr hohe Korrelation und Übereinstimmung zwischen vollautomatischer und halbautomatischer Kalziumbewertungssoftware.

• Weniger zeitaufwendig als herkömmliche halbautomatische Verfahren.

• Hervorragendes Instrument zur arterienspezifischen Kalziumbestimmung im klinischen Alltag.

Key words

computed tomography - calcium scoring - coronary artery calcium - diagnostic accuracy - machine learning

Publication History

Received: 01 October 2021

Accepted: 21 November 2021

Article published online:
26 January 2022

• References

• 1 Nowbar AN, Gitto M, Howard JP. et al. Mortality From Ischemic Heart Disease. Circ Cardiovasc Qual Outcomes 2019; 12: e005375
• 2 Hecht HS. Coronary artery calcium scanning: past, present, and future. JACC Cardiovascular imaging 2015; 8: 579-596
• 3 Lee H, Martin S, Burt JR. et al. Machine Learning and Coronary Artery Calcium Scoring. Curr Cardiol Rep 2020; 22: 90
• 4 Agatston AS, Janowitz WR, Hildner FJ. et al. Quantification of coronary artery calcium using ultrafast computed tomography. Journal of the American College of Cardiology 1990; 15: 827-832
• 5 Divakaran S, Cheezum MK, Hulten EA. et al. Use of cardiac CT and calcium scoring for detecting coronary plaque: implications on prognosis and patient management. The British journal of radiology 2015; 88: 20140594
• 6 Piepoli MF, Hoes AW, Agewall S. et al. 2016 European Guidelines on cardiovascular disease prevention in clinical practice: The Sixth Joint Task Force of the European Society of Cardiology and Other Societies on Cardiovascular Disease Prevention in Clinical Practice (constituted by representatives of 10 societies and by invited experts) Developed with the special contribution of the European Association for Cardiovascular Prevention & Rehabilitation (EACPR). Atherosclerosis 2016; 252: 207-274
• 7 Greenland P, Alpert J, Beller G. et al. American HeartAsscoiation Tasl Force on Practice Guidelines. 2010 ACCF/AHA guideline for assessment of cardiovascular risk in asymptomatic adults: executive summary: a report of the American College of Cardiology Foundation/American Heart Association Task Force on Practice Guidelines. Circulation 2010; 122: 2748-2764
• 8 Alluri K, Joshi PH, Henry TS. et al. Scoring of coronary artery calcium scans: history, assumptions, current limitations, and future directions. Atherosclerosis 2015; 239: 109-117
• 9 Rumberger JA, Kaufman L. A Rosetta Stone for Coronary Calcium Risk Stratification: Agatston, Volume, and Mass Scores in 11,490 Individuals. American Journal of Roentgenology 2003; 181: 743-748
• 10 Sandstedt M, Henriksson L, Janzon M. et al. Evaluation of an AI-based, automatic coronary artery calcium scoring software. European radiology 2020; 30: 1671-1678
• 11 Wolterink JM, Leiner T, De Vos BD. et al. An evaluation of automatic coronary artery calcium scoring methods with cardiac CT using the orCaScore framework. Medical physics 2016; 43: 2361-2373
• 12 Brunner G, Chittajallu DR, Kurkure U. et al. Toward the automatic detection of coronary artery calcification in non-contrast computed tomography data. The international journal of cardiovascular imaging 2010; 26: 829-838
• 13 Sharma P, Suehling M, Flohr T. et al. Artificial Intelligence in Diagnostic Imaging: Status Quo, Challenges, and Future Opportunities. J Thorac Imaging 2020; 35: S11-S16
• 14 Martin SS, Assen Mv, Rapaka S. et al. Evaluation of a Deep Learning – 2013;Based Automated CT Coronary Artery Calcium Scoring Algorithm. JACC: Cardiovascular Imaging 2020; 13: 524-526
• 15 Pesapane F, Codari M, Sardanelli F. Artificial intelligence in medical imaging: threat or opportunity? Radiologists again at the forefront of innovation in medicine. Eur Radiol Exp 2018; 2: 35
• 16 Tota-Maharaj R, Joshi PH, Budoff MJ. et al. Usefulness of regional distribution of coronary artery calcium to improve the prediction of all-cause mortality. Am J Cardiol 2015; 115: 1229-1234
• 17 Blaha MJ, Budoff MJ, Tota-Maharaj R. et al. Improving the CAC Score by Addition of Regional Measures of Calcium Distribution: Multi-Ethnic Study of Atherosclerosis. JACC Cardiovascular imaging 2016; 9: 1407-1416
• 18 Greenland P, Bonow RO, Brundage BH. et al. ACCF/AHA 2007 Clinical Expert Consensus Document on Coronary Artery Calcium Scoring By Computed Tomography in Global Cardiovascular Risk Assessment and in Evaluation of Patients With Chest Pain: A Report of the American College of Cardiology Foundation Clinical Expert Consensus Task Force (ACCF/AHA Writing Committee to Update the 2000 Expert Consensus Document on Electron Beam Computed Tomography) Developed in Collaboration With the Society of Atherosclerosis Imaging and Prevention and the Society of Cardiovascular Computed Tomography. Journal of the American College of Cardiology 2007; 49: 378-402
• 19 Datta D. blandr: a Bland-Altman Method Comparison package for R. Zenodo doi 2017; 10
• 20 Shahzad R, van Walsum T, Schaap M. et al. Vessel Specific Coronary Artery Calcium Scoring: An Automatic System. Academic radiology 2013; 20: 1-9
• 21 Blaha MJ, Mortensen MB, Kianoush S. et al. Coronary Artery Calcium Scoring. JACC: Cardiovascular Imaging 2017; 10: 923-937
• 22 Willemink MJ, Vliegenthart R, Takx RA. et al. Coronary artery calcification scoring with state-of-the-art CT scanners from different vendors has substantial effect on risk classification. Radiology 2014; 273: 695-702
• 23 Weininger M, Ritz KS, Schoepf UJ. et al. Interplatform reproducibility of CT coronary calcium scoring software. Radiology 2012; 265: 70-77
• 24 van Assen M, Martin SS, Varga-Szemes A. et al. Automatic coronary calcium scoring in chest CT using a deep neural network in direct comparison with non-contrast cardiac CT: A validation study. European journal of radiology 2021; 134: 109428
• 25 Lessmann N, Išgum I, Setio AA. et al., editors. Deep convolutional neural networks for automatic coronary calcium scoring in a screening study with low-dose chest CT. Medical Imaging 2016: Computer-Aided Diagnosis; 2016: International Society for Optics and Photonics.
• 26 Velzen SGMv, Lessmann N, Velthuis BK. et al. Deep Learning for Automatic Calcium Scoring in CT: Validation Using Multiple Cardiac CT and Chest CT Protocols. Radiology 2020; 295: 66-79