Validation of Multiparametric Ultrasonography Criteria with Digital Subtraction Angiography in Carotid Artery Disease: A Prospective Multicenter Study^[*]

Abstract

Purpose The German Society of Ultrasound in Medicine (DEGUM) recently revised its multiparametric criteria for duplex ultrasonography (DUS) grading of internal carotid artery (ICA) disease. We determined the diagnostic accuracy of the revised DEGUM criteria for ultrasonography grading of ICA disease in a prospective multicenter study.

Materials and Methods We evaluated consecutive patients who underwent digital subtraction angiography of the extracranial carotid arteries at four tertiary care hospitals. Blinded investigators graded ICA disease according to DEGUM-recommended ultrasonography criteria and calculated NASCET-type percent stenosis from angiography images. Endpoints included overall classification accuracy, prediction of clinically relevant disease categories and between-test agreement in the continuous range of percent stenosis.

Results A total of 121 patients (median age: 69 [IQR, 16] years; 74 % men; median time between DUS and angiography: 1 day [IQR, 2]) provided 163 DUS-angiography carotid artery pairs. The classification accuracy of the DEGUM criteria to predict stenosis within 10 % increments as compared to angiography was 34.9 % (95 % CI, 28.0 – 42.6). The sensitivity of DUS for the detection of moderate (50 – 69 %) and severe (70 – 99 %) stenosis was 35 % and 81 %, with an overall accuracy of 73 % and 74 %, respectively. The specificity was 89 % and 69 %, respectively. Considering the continuous spectrum of the disease (0 – 100 %), the Bland-Altman interval limit of agreement was 51 %.

Conclusion At laboratories experienced with ultrasound grading of the extracranial ICA, the revised DEGUM multiparametric ultrasonography criteria do not eliminate the need for a confirmatory test for the identification of clinically relevant grades of the disease.

Zusammenfassung

Ziel Die multiparametrischen Ultraschallkriterien zur Graduierung von Karotisstenosen der Deutschen Gesellschaft für Ultraschall in der Medizin (DEGUM) wurden revidiert, bislang aber nicht mit dem Goldstandard Angiografie validiert. Ziel dieser prospektiven multizentrischen Validierungsstudie war es, die diagnostische Güte der DEGUM Ultraschallkriterien zu bestimmen.

Material und Methoden An vier Zentren wurden konsekutive Patienten eingeschlossen, bei denen routinemäßig eine digitale Subtraktionsangiografie (DSA) der extrakraniellen Arteria carotis interna (ACI) durchgeführt wurde. Die Graduierung des Stenosegrades der ACI erfolgte verblindet gemäß der multiparametrischen DEGUM Ultraschallkriterien. Angiografisch erfolgte die verblindete Stenosegraduierung entsprechend der NASCET Definition. Endpunkte umfassten die Gesamttrefferquote zur Vorhersage von Stenosen in vorgegebenen DEGUM Stenoseintervallen, Vorhersage von klinisch relevanten Stenosekategorien und diagnostische Übereinstimmung beider Methoden im kontinuierlichen Stenosebereich.

Ergebnisse Von insgesamt 121 rekrutierten Patienten (mittleres Alter, 69 [IQR, 16] Jahre, 74 % Männer, mediane Zeit zwischen Duplexsonografie und DSA, 1 Tag [IQR, 2]) ergaben sich 163 Duplex-DSA Gefäßpaare für die Validierung. Die Gesamttrefferquote der DEGUM-Ultraschallkriterien zur Vorhersage des Stenosegrades innerhalb von 10 % -Schritten im Vergleich zur DSA betrug 34,9 % (95 % CI, 28,0 – 42,6). Die Sensitivität der Ultraschallkriterien für den Nachweis von mittelgradigen (50 – 69 %) und hochgradigen (70 – 99 %) Stenosen betrug 35 % und 81 % mit einer Gesamtgenauigkeit von 73 % und 74 %. Die Spezifität betrug 89 % und 69 %. Der Bland-Altman-Übereinstimmungsbereich zwischen beiden Methoden betrug 51 %.

Schlussfolgerung In der Identifikation von klinisch relevanten Stenosekategorien der ACI beseitigen die multiparametrischen DEGUM Ultraschallkriterien nicht die Notwendigkeit eines vaskulären Bestätigungstestes.

Key words

angiography - ultrasound - carotid artery disease - diagnostic testing

^* This study was funded by the Roland-Ernst-Foundation (Project # 6/14)

• References

• 1 Barnett HJ, Taylor DW, Eliasziw M. et al. Benefit of carotid endarterectomy in patients with symptomatic moderate or severe stenosis. North American Symptomatic Carotid Endarterectomy Trial Collaborators. N Engl J Med 1998; 339: 1415-1425
• 2 European Carotid Surgery Trialists’ Collaborative Group. Randomised trial of endarterectomy for recently symptomatic carotid stenosis: final results of the MRC European Carotid Surgery Trial (ECST). Lancet 1998; 351: 1379-1387
• 3 Halliday A, Harrison M, Hayter E. et al. 10-Year stroke prevention after successful carotid endarterectomy for asymptomatic stenosis (ACST-1): a multicentre randomized trial. Lancet 2010; 376: 1074-1084
• 4 Muluk SC, Muluk VS, Sugimoto H. et al. Progression of asymptomatic carotid stenosis: a natural history study in 1004 patients. J Vasc Surg 1999; 29: 208-214
• 5 Rosenfield K, Matsumura JS, Chaturvedi S. et al. Randomized Trial of Stent versus Surgery for Asymptomatic Carotid Stenosis. N Engl J Med 2016; 374: 1011-1020
• 6 Barlinn K, Alexandrov AV. Vascular imaging in stroke: comparative analysis. Neurotherapeutics 2011; 8: 340-348
• 7 Beach KW, Leotta DF, Zierler RE. Carotid Doppler velocity measurements and anatomic stenosis: correlation is futile. Vasc Endovascular Surg 2012; 46: 466-474
• 8 Mozzini C, Roscia G, Casadei A. et al. Searching the perfect ultrasonic classification in assessing carotid artery stenosis: comparison and remarks upon the existing ultrasound criteria. J Ultrasound 2016; 19: 83-90
• 9 Gray C, Murray SA, Connolly M. et al. The implications of different criteria for grading carotid artery stenosis by duplex ultrasound. J Clin Ultrasound 2015; 43: 563-566
• 10 Arning C, Widder B, von Reutern GM. et al. Revision of DEGUM ultrasound criteria for grading internal carotid artery stenoses and transfer to NASCET measurement. Ultraschall in Med 2010; 31: 251-257
• 11 Klingelhöfer J. Ultrasonography of carotid stenosis. Int J Clin Neurosci Ment Health 2014; 1: S04
• 12 von Reutern GM, Goertler MW, Bornstein NM. et al. Grading carotid stenosis using ultrasonic methods. Stroke 2012; 43: 916-921
• 13 Alexandrov AV, Needleman L. Carotid artery stenosis: making complex assessments of a simple problem or simplifying approach to a complex disease?. Stroke 2012; 43: 627-628
• 14 Bossuyt PM, Reitsma JB, Bruns DE. et al. STARD 2015: An Updated List of Essential Items for Reporting Diagnostic Accuracy Studies. BMJ 2015; 351: h5527
• 15 Lewis JR, Sauro J. When 100% really isn’t 100%: improving the accuracy of small-sample estimates of completion rates. J Usability Studies 2006; 1: 136-150
• 16 Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1986; i: 307-310
• 17 Cohen J. Weighted kappa: nominal scale agreement with provision for scaled disagreement or partial credit. Psychol Bull 1968; 70: 213-220
• 18 Shrout PE, Fleiss JL. Intraclass correlations: uses in assessing rater reliability. Psychol Bull 1979; 86: 420-428
• 19 Grant EG, Benson CB, Moneta GL. et al. Carotid artery stenosis: gray-scale and Doppler US diagnosis–Society of Radiologists in Ultrasound Consensus Conference. Radiology 2003; 229: 340-346
• 20 Oates CP, Naylor AR, Hartshorne T. et al. Joint recommendations for reporting carotid ultrasound investigations in the United Kingdom. Eur J Vasc Endovasc Surg 2009; 251-261
• 21 Beach KW, Bergelin RO, Leotta DF. et al. Standardized ultrasound evaluation of carotid stenosis for clinical trials: University of Washington Ultrasound Reading Center. Cardiovasc Ultrasound 2010; 8: 39
• 22 Hathout GM, Fink JR, El-Saden SM. et al. Sonographic NASCET index: a new doppler parameter for assessment of internal carotid artery stenosis. AJNR Am J Neuroradiol 2005; 26: 68-75
• 23 Wardlaw JM, Chappell FM, Best JJ. et al. Non-invasive imaging compared with intra-arterial angiography in the diagnosis of symptomatic carotid stenosis: a metaanalysis. Lancet 2006; 367: 1503-1512
• 24 Chappell FM, Wardlaw JM, Young GR. et al. Carotid artery stenosis: accuracy of noninvasive tests—individual patient data meta-analysis. Radiology 2009; 251: 493-502
• 25 Rafailidis V, Charitanti A, Tegos T. et al. Contrast-enhanced ultrasound of the carotid system: a review of the current literature. J Ultrasound 2017; 20: 97-109
• 26 Patel SG, Collie DA, Wardlaw JM. Outcome, observer reliability, and patient preferences if CTA, MRA, or Doppler ultrasound were used, individually or together, instead of digital subtraction angiography before carotid endarterectomy. J Neurol Neurosurg Psychiatry 2002; 73: 21-28
• 27 Mylonas SN, Antonopoulos CN, Moulakakis KG. et al. Management of Patients with Internal Carotid Artery Near-total Occlusion: An Updated Meta-analysis. Ann Vasc Surg 2015; 29: 1664-1672
• 28 Fox AJ. How to measure carotid stenosis. Radiology 1993; 186: 316-318
• 29 Kohn MA, Carpenter CR, Newman TB. Understanding the direction of bias in studies of diagnostic test accuracy. Acad Emerg Med 2013; 20: 1194-1206
• 30 Matz O, Nikoubashman O, Rajkumar P. et al. Grading of proximal internal carotid artery (ICA) stenosis by Doppler/duplex ultrasound (DUS) and computed tomographic angiography (CTA): correlation and interrater reliability in real-life practice. Acta Neurol Belg 2017; 117: 183-188
• 31 Müller M, Agten CA, Österreich M. et al. Assessing internal carotid artery stenosis with a semiautomated computed tomography angiography tool and duplex ultrasound. J Vasc Surg 2015; 61: 1449-1456
• 32 Barlinn K, Floegel T, Kitzler HH. et al. Multi-parametric ultrasound criteria for internal carotid artery disease-comparison with CT angiography. Neuroradiology 2016; 58: 845-851