Artificial intelligence-assisted system for the assessment of Forrest classification of peptic ulcer bleeding: a multicenter diagnostic study

Abstract

Background Inaccurate Forrest classification may significantly affect clinical outcomes, especially in high risk patients. Therefore, this study aimed to develop a real-time deep convolutional neural network (DCNN) system to assess the Forrest classification of peptic ulcer bleeding (PUB).

Methods A training dataset (3868 endoscopic images) and an internal validation dataset (834 images) were retrospectively collected from the 900th Hospital, Fuzhou, China. In addition, 521 images collected from four other hospitals were used for external validation. Finally, 46 endoscopic videos were prospectively collected to assess the real-time diagnostic performance of the DCNN system, whose diagnostic performance was also prospectively compared with that of three senior and three junior endoscopists.

Results The DCNN system had a satisfactory diagnostic performance in the assessment of Forrest classification, with an accuracy of 91.2% (95%CI 89.5%–92.6%) and a macro-average area under the receiver operating characteristic curve of 0.80 in the validation dataset. Moreover, the DCNN system could judge suspicious regions automatically using Forrest classification in real-time videos, with an accuracy of 92.0% (95%CI 80.8%–97.8%). The DCNN system showed more accurate and stable diagnostic performance than endoscopists in the prospective clinical comparison test. This system helped to slightly improve the diagnostic performance of senior endoscopists and considerably enhance that of junior endoscopists.

Conclusion The DCNN system for the assessment of the Forrest classification of PUB showed satisfactory diagnostic performance, which was slightly superior to that of senior endoscopists. It could therefore effectively assist junior endoscopists in making such diagnoses during gastroscopy.

Publication History

Received: 03 February 2023

Accepted after revision: 10 January 2024

Article published online:
27 February 2024

• References

• 1 Chang-Chien CS, Wu CS, Chen PC. et al. Different implications of stigmata of recent hemorrhage in gastric and duodenal ulcers. Dig Dis Sci 1988; 33: 400-404
• 2 Lau JY, Sung JJ, Chan AC. et al. Stigmata of hemorrhage in bleeding peptic ulcers: an interobserver agreement study among international experts. Gastrointest Endosc 1997; 46: 33-36
• 3 Kate V, Sureshkumar S, Gurushankari B. et al. Acute upper non-variceal and lower gastrointestinal bleeding. J Gastrointest Surg 2022; 26: 932-949
• 4 de Groot NL, van Oijen MGH, Kessels K. et al. Reassessment of the predictive value of the Forrest classification for peptic ulcer rebleeding and mortality: can classification be simplified?. Endoscopy 2014; 46: 46-52
• 5 Bleau BL, Gostout CJ, Sherman KE. et al. Recurrent bleeding from peptic ulcer associated with adherent clot: a randomized study comparing endoscopic treatment with medical therapy. Gastrointest Endosc 2002; 56: 1-6
• 6 Heldwein W, Schreiner J, Pedrazzoli J. et al. Is the Forrest classification a useful tool for planning endoscopic therapy of bleeding peptic ulcers?. Endoscopy 1989; 21: 258-262
• 7 Laine L, Freeman M, Cohen H. Lack of uniformity in evaluation of endoscopic prognostic features of bleeding ulcers. Gastrointest Endosc 1994; 40: 411-417
• 8 Laine L, Stein C, Sharma V. A prospective outcome study of patients with clot in an ulcer and the effect of irrigation. Gastrointest Endosc 1996; 43: 107-110
• 9 Guglielmi A, Ruzzenente A, Sandri M. et al. Risk assessment and prediction of rebleeding in bleeding gastroduodenal ulcer. Endoscopy 2002; 34: 778-786
• 10 Tang D, Wang L, Jiang J. et al. A novel deep learning system for diagnosing early esophageal squamous cell carcinoma: a multicenter diagnostic study. Clin Transl Gastroenterol 2021; 12: e00393
• 11 Luo Y, Zhang Y, Liu M. et al. Artificial intelligence-assisted colonoscopy for detection of colon polyps: a prospective, randomized cohort study. J Gastrointest Surg 2021; 25: 2011-2018
• 12 Tang D, Wang L, Ling T. et al. Development and validation of a real-time artificial intelligence-assisted system for detecting early gastric cancer: a multicentre retrospective diagnostic study. EBioMedicine 2020; 62: 103146
• 13 Nakashima H, Kawahira H, Kawachi H. et al. Artificial intelligence diagnosis of Helicobacter pylori infection using blue laser imaging-bright and linked color imaging: a single-center prospective study. Ann Gastroenterol 2018; 31: 462-468
• 14 Zhu Y, Wang QC, Xu MD. et al. Application of convolutional neural network in the diagnosis of the invasion depth of gastric cancer based on conventional endoscopy. Gastrointest Endosc 2019; 89: 806-815 e801
• 15 Aoki T, Yamada A, Aoyama K. et al. Automatic detection of erosions and ulcerations in wireless capsule endoscopy images based on a deep convolutional neural network. Gastrointest Endosc 2019; 89: 357-363 e352
• 16 Shung DL, Au B, Taylor RA. et al. Validation of a machine learning model that outperforms clinical risk scoring systems for upper gastrointestinal bleeding. Gastroenterology 2020; 158: 160-167
• 17 Seo DW, Yi H, Park B. et al. Prediction of adverse events in stable non-variceal gastrointestinal bleeding using machine learning. J Clin Med 2020; 9: 1-15
• 18 Yen H-H, Wu P-Y, Su P-Y. et al. Performance comparison of the deep learning and the human endoscopist for bleeding peptic ulcer disease. J Med Biol Eng 2021; 41: 504-513
• 19 Forrest JA, Finlayson ND, Shearman DJ. Endoscopy in gastrointestinal bleeding. Lancet 1974; 2: 394-397
• 20 Wang C-Y, Bochkovskiy A, Liao H-YM. YOLOv7: trainable bag-of-freebies sets new state-of-the-art for real-time object detectors. Accessed January 18, 2024 at: https://arxiv.org/abs/2207.02696v1
• 21 Shimamoto Y, Ishihara R, Kato Y. et al. Real-time assessment of video images for esophageal squamous cell carcinoma invasion depth using artificial intelligence. J Gastroenterol 2020; 55: 1037-1045
• 22 Yen HH, Wu PY, Chen MF. et al. Current status and future perspective of artificial intelligence in the management of peptic ulcer bleeding: a review of recent literature. J Clin Med 2021; 10: 1-10