PDF herunterladen
CC BY-NC-ND 4.0 · Indian J Radiol Imaging 2024; 34(02): 269-275
DOI: 10.1055/s-0043-1777289
Original Article

Radiological Differential Diagnoses Based on Cardiovascular and Thoracic Imaging Patterns: Perspectives of Four Large Language Models

Pradosh Kumar Sarangi 
1   Department of Radiodiagnosis, All India Institute of Medical Sciences, Deoghar, Jharkhand, India
,
Aparna Irodi 
2   Department of Radiodiagnosis, Christian Medical College and Hospital, Vellore, Tamil Nadu, India
,
Swaha Panda 
3   Department of Otorhinolaryngology and Head and Neck Surgery, All India Institute of Medical Sciences, Deoghar, Jharkhand, India
,
Debasish Swapnesh Kumar Nayak 
4   Department of Computer Science and Engineering, Siksha ‘O’ Anusandhan (Deemed to be) University, Bhubaneswar, Odisha, India
,
Himel Mondal 
5   Department of Physiology, All India Institute of Medical Sciences, Deoghar, Jharkhand, India
› Institutsangaben Funding None.
› Weitere Informationen
Auch verfügbar auf
   Lizenzen und Reprints

Abstract

Background Differential diagnosis in radiology is a critical aspect of clinical decision-making. Radiologists in the early stages may find difficulties in listing the differential diagnosis from image patterns. In this context, the emergence of large language models (LLMs) has introduced new opportunities as these models have the capacity to access and contextualize extensive information from text-based input.

Objective The objective of this study was to explore the utility of four LLMs—ChatGPT3.5, Google Bard, Microsoft Bing, and Perplexity—in providing most important differential diagnoses of cardiovascular and thoracic imaging patterns.

Methods We selected 15 unique cardiovascular (n = 5) and thoracic (n = 10) imaging patterns. We asked each model to generate top 5 most important differential diagnoses for every pattern. Concurrently, a panel of two cardiothoracic radiologists independently identified top 5 differentials for each case and came to consensus when discrepancies occurred. We checked the concordance and acceptance of LLM-generated differentials with the consensus differential diagnosis. Categorical variables were compared by binomial, chi-squared, or Fisher's exact test.

Results A total of 15 cases with five differentials generated a total of 75 items to analyze. The highest level of concordance was observed for diagnoses provided by Perplexity (66.67%), followed by ChatGPT (65.33%) and Bing (62.67%). The lowest score was for Bard with 45.33% of concordance with expert consensus. The acceptance rate was highest for Perplexity (90.67%), followed by Bing (89.33%) and ChatGPT (85.33%). The lowest acceptance rate was for Bard (69.33%).

Conclusion Four LLMs—ChatGPT3.5, Google Bard, Microsoft Bing, and Perplexity—generated differential diagnoses had high level of acceptance but relatively lower concordance. There were significant differences in acceptance and concordance among the LLMs. Hence, it is important to carefully select the suitable model for usage in patient care or in medical education.

Keywords

artificial intelligence - cardiothoracic - ChatGPT - Google Bard - Microsoft Bing - perplexity - differential diagnosis - radiologists


Publikationsverlauf

Artikel online veröffentlicht:
28. Dezember 2023

© 2023. Indian Radiological Association. This is an open access article published by Thieme under the terms of the Creative Commons Attribution-NonDerivative-NonCommercial License, permitting copying and reproduction so long as the original work is given appropriate credit. Contents may not be used for commercial purposes, or adapted, remixed, transformed or built upon. (https://creativecommons.org/licenses/by-nc-nd/4.0/)

Thieme Medical and Scientific Publishers Pvt. Ltd.
A-12, 2nd Floor, Sector 2, Noida-201301 UP, India

 

  • References

  • 1 Hussain S, Mubeen I, Ullah N. et al. Modern diagnostic imaging technique applications and risk factors in the medical field: a review. BioMed Res Int 2022; 2022: 5164970
    CrossrefPubMedGoogle Scholar
  • 2 Alberts IL, Mercolli L, Pyka T. et al. Large language models (LLM) and ChatGPT: what will the impact on nuclear medicine be?. Eur J Nucl Med Mol Imaging 2023; 50 (06) 1549-1552
    CrossrefPubMedGoogle Scholar
  • 3 De Angelis L, Baglivo F, Arzilli G. et al. ChatGPT and the rise of large language models: the new AI-driven infodemic threat in public health. Front Public Health 2023; 11: 1166120
    CrossrefPubMedGoogle Scholar
  • 4 Kumari A, Kumari A, Singh A. et al. Large language models in hematology case solving: a comparative study of ChatGPT-3.5, Google Bard, and Microsoft Bing. Cureus 2023; 15 (08) e43861
    PubMedGoogle Scholar
  • 5 Thirunavukarasu AJ, Ting DSJ, Elangovan K, Gutierrez L, Tan TF, Ting DSW. Large language models in medicine. Nat Med 2023; 29 (08) 1930-1940
    CrossrefPubMedGoogle Scholar
  • 6 Rao A, Kim J, Kamineni M, Pang M, Lie W, Dreyer KJ, Succi MD. Evaluating GPT as an adjunct for radiologic decision making: GPT-4 versus GPT-3.5 in a breast imaging pilot. J Am Coll Radiol 2023; 20 (10) 990-997
    CrossrefPubMedGoogle Scholar
  • 7 Bhayana R, Krishna S, Bleakney RR. Performance of ChatGPT on a radiology board-style examination: insights into current strengths and limitations. Radiology 2023; 307 (05) e230582
    CrossrefPubMedGoogle Scholar
  • 8 Currie G, Singh C, Nelson T, Nabasenja C, Al-Hayek Y, Spuur K. ChatGPT in medical imaging higher education. Radiography 2023; 29 (04) 792-799
    CrossrefPubMedGoogle Scholar
  • 9 Kottlors J, Bratke G, Rauen P. et al. Feasibility of differential diagnosis based on imaging patterns using a large language model. Radiology 2023; 308 (01) e231167
    CrossrefPubMedGoogle Scholar
  • 10 Davies SG. Chapman & Nakielny's Aids to Radiological Differential Diagnosis. 6th ed.. Edinburg: Elsevier Saunders; 2014
    Google Scholar
  • 11 Elkassem AA, Smith AD. Potential use cases for ChatGPT in radiology reporting. AJR Am J Roentgenol 2023; 221 (03) 373-376
    CrossrefPubMedGoogle Scholar
  • 12 Schukow C, Smith SC, Landgrebe E. et al. Application of ChatGPT in routine diagnostic pathology: promises, pitfalls, and potential future directions. Adv Anat Pathol 2023; (e-pub ahead of print) DOI: 10.1097/PAP.0000000000000406.
    CrossrefPubMedGoogle Scholar
  • 13 Liu J, Wang C, Liu S. Utility of ChatGPT in clinical practice. J Med Internet Res 2023; 25: e48568
    CrossrefPubMedGoogle Scholar
  • 14 Mondal H, Mondal S, Podder I. Using ChatGPT for writing articles for patients' education for dermatological diseases: a pilot study. Indian Dermatol Online J 2023; 14 (04) 482-486
    CrossrefPubMedGoogle Scholar
  • 15 Tsang R. Practical applications of ChatGPT in undergraduate medical education. J Med Educ Curric Dev 2023; 10: 23 821205231178449
    CrossrefPubMedGoogle Scholar