PDF herunterladen
CC BY-NC-ND 4.0 · J Neurol Surg Rep 2024; 85(03): e118-e123
DOI: 10.1055/a-2358-8928
Original Article

Development of a Machine Learning–Enabled Virtual Reality Tool for Preoperative Planning of Functional Endoscopic Sinus Surgery

Varun Gudapati 
1   David Geffen School of Medicine, UCLA, Los Angeles, California, United States
,
Alexander Chen
1   David Geffen School of Medicine, UCLA, Los Angeles, California, United States
,
Scott Meyer
1   David Geffen School of Medicine, UCLA, Los Angeles, California, United States
,
Chung-Chieh Jay Kuo
2   Ming-Hsieh Department of Electrical Engineering, USC, Los Angeles, California, United States
,
Yichen Ding
1   David Geffen School of Medicine, UCLA, Los Angeles, California, United States
,
Tzung K. Hsiai
1   David Geffen School of Medicine, UCLA, Los Angeles, California, United States
,
Marilene Wang
1   David Geffen School of Medicine, UCLA, Los Angeles, California, United States
› Institutsangaben
› Weitere Informationen
Auch verfügbar auf
   Lizenzen und Reprints

Abstract

Objectives Virtual reality (VR) is an increasingly valuable teaching tool, but current simulators are not typically clinically scalable due to their reliance on inefficient manual segmentation. The objective of this project was to leverage a high-throughput and accurate machine learning method to automate data preparation for a patient-specific VR simulator used to explore preoperative sinus anatomy.

Methods An endoscopic VR simulator was designed in Unity to enable interactive exploration of sinus anatomy. The Saak transform, a data-efficient machine learning method, was adapted to accurately segment sinus computed tomography (CT) scans using minimal training data, and the resulting data were reconstructed into three-dimensional (3D) patient-specific models that could be explored in the simulator.

Results Using minimal training data, the Saak transform–based machine learning method offers accurate soft-tissue segmentation. When explored with an endoscope in the VR simulator, the anatomical models generated by the algorithm accurately capture key sinus structures and showcase patient-specific variability in anatomy.

Conclusion By offering an automatic means of preparing VR models from a patient's raw CT scans, this pipeline takes a key step toward clinical scalability. In addition to preoperative planning, this system also enables virtual endoscopy—a tool that is particularly useful in the COVID-19 era. As VR technology inevitably continues to develop, such a foundation will help ensure that future innovations remain clinically accessible.

Keywords

computed tomography - endoscopic sinus surgery - imaging - simulation training - FESS


Publikationsverlauf

Eingereicht: 05. März 2024

Angenommen: 16. Mai 2024

Accepted Manuscript online:
02. Juli 2024

Artikel online veröffentlicht:
05. August 2024

© 2024. The Author(s). This is an open access article published by Thieme under the terms of the Creative Commons Attribution-NonDerivative-NonCommercial License, permitting copying and reproduction so long as the original work is given appropriate credit. Contents may not be used for commercial purposes, or adapted, remixed, transformed or built upon. (https://creativecommons.org/licenses/by-nc-nd/4.0/)

Georg Thieme Verlag KG
Rüdigerstraße 14, 70469 Stuttgart, Germany

 

  • References

  • 1 Khariwala SS, Weinreich HM, McCoul ED. et al. Leveraging COVID-19-inspired changes to advance otolaryngology-here to stay. JAMA Otolaryngol Head Neck Surg 2020; 146 (07) 605-607
    CrossrefPubMedGoogle Scholar
  • 2 Singh RP, Javaid M, Kataria R, Tyagi M, Haleem A, Suman R. Significant applications of virtual reality for COVID-19 pandemic. Diabetes Metab Syndr 2020; 14 (04) 661-664
    CrossrefPubMedGoogle Scholar
  • 3 El Beheiry M, Doutreligne S, Caporal C, Ostertag C, Dahan M, Masson JB. Virtual reality: beyond visualization. J Mol Biol 2019; 431 (07) 1315-1321
    CrossrefPubMedGoogle Scholar
  • 4 Abiri A, Ding Y, Abiri P. et al. Simulating developmental cardiac morphology in virtual reality using a deformable image registration approach. Ann Biomed Eng 2018; 46 (12) 2177-2188
    CrossrefPubMedGoogle Scholar
  • 5 Ding Y, Abiri A, Abiri P. et al. Integrating light-sheet imaging with virtual reality to recapitulate developmental cardiac mechanics. JCI Insight 2017; 2 (22) e97180
    CrossrefPubMedGoogle Scholar
  • 6 Mafee MF, Chow JM, Meyers R. Functional endoscopic sinus surgery: anatomy, CT screening, indications, and complications. AJR Am J Roentgenol 1993; 160 (04) 735-744
    CrossrefPubMedGoogle Scholar
  • 7 Hosemann W. Comprehensive review on danger points, complications and medico-legal aspects in endoscopic sinus surgery. Laryngorhinootologie 2013; 92 (Suppl. 01) S88-S136
    PubMedGoogle Scholar
  • 8 Hosemann W, Grimm A. Surgical anatomy of the maxillary sinus. HNO 2020; 68 (08) 555-565
    PubMedGoogle Scholar
  • 9 Beswick DM, Ramakrishnan VR. The utility of image guidance in endoscopic sinus surgery: a narrative review. JAMA Otolaryngol Head Neck Surg 2020; 146 (03) 286-290
    CrossrefPubMedGoogle Scholar
  • 10 Casiano RR, Numa Jr WA. Efficacy of computed tomographic image–guided endoscopic sinus surgery in residency training programs. Laryngoscope 2000; 110 (08) 1277-1282
    CrossrefPubMedGoogle Scholar
  • 11 Fried MP, Parikh SR, Sadoughi B. Image-guidance for endoscopic sinus surgery. Laryngoscope 2008; 118 (07) 1287-1292
    CrossrefPubMedGoogle Scholar
  • 12 Casiano RR. Intraoperative image-guidance technology. Arch Otolaryngol Head Neck Surg 1999; 125 (11) 1275-1278
    CrossrefPubMedGoogle Scholar
  • 13 Rudman DT, Stredney D, Sessanna D. et al. Functional endoscopic sinus surgery training simulator. Laryngoscope 1998; 108 (11, Pt 1): 1643-1647
    CrossrefPubMedGoogle Scholar
  • 14 Arora A, Lau LY, Awad Z, Darzi A, Singh A, Tolley N. Virtual reality simulation training in otolaryngology. Int J Surg 2014; 12 (02) 87-94
    CrossrefPubMedGoogle Scholar
  • 15 Andersen SA, Konge L, Cayé-Thomasen P, Sørensen MS. Retention of mastoidectomy skills after virtual reality simulation training. JAMA Otolaryngol Head Neck Surg 2016; 142 (07) 635-640
    CrossrefPubMedGoogle Scholar
  • 16 Barber SR, Jain S, Son YJ, Chang EH. Virtual functional endoscopic sinus surgery simulation with 3D-printed models for mixed-reality nasal endoscopy. Otolaryngol Head Neck Surg 2018; 159 (05) 933-937
    CrossrefPubMedGoogle Scholar
  • 17 Kuppersmith RB, Johnston R, Jones SB, Jenkins HA. Virtual reality surgical simulation and otolaryngology. Arch Otolaryngol Head Neck Surg 1996; 122 (12) 1297-1298
    CrossrefPubMedGoogle Scholar
  • 18 Fried MP, Satava R, Weghorst S. et al. Identifying and reducing errors with surgical simulation. Qual Saf Health Care 2004; 13 (Suppl. 01) i19-i26
    CrossrefPubMedGoogle Scholar
  • 19 Fried MP, Moharir VM, Shinmoto H. et al. Virtual laryngoscopy. Ann Otol Rhinol Laryngol 1999; 108 (03) 221-226
    CrossrefPubMedGoogle Scholar
  • 20 Hsieh TY, Cervenka B, Dedhia R, Strong EB, Steele T. Assessment of a patient-specific, 3-dimensionally printed endoscopic sinus and skull base surgical model. JAMA Otolaryngol Head Neck Surg 2018; 144 (07) 574-579
    CrossrefPubMedGoogle Scholar
  • 21 VanKoevering KK, Hollister SJ, Green GE. Advances in 3-dimensional printing in otolaryngology: a review. JAMA Otolaryngol Head Neck Surg 2017; 143 (02) 178-183
    CrossrefPubMedGoogle Scholar
  • 22 VanKoevering KK, Malloy KM. Emerging role of three-dimensional printing in simulation in otolaryngology. Otolaryngol Clin North Am 2017; 50 (05) 947-958
    CrossrefPubMedGoogle Scholar
  • 23 Hsieh TY, Cervenka B, Steele TO. Validation of haptic properties of materials for endoscopic sinus and skull base surgery simulation: reply. JAMA Otolaryngol Head Neck Surg 2018; 144 (12) 1185-1186
    CrossrefPubMedGoogle Scholar
  • 24 Gilani S, Norbash AM, Ringl H, Rubin GD, Napel S, Terris DJ. Virtual endoscopy of the paranasal sinuses using perspective volume rendered helical sinus computed tomography. Laryngoscope 1997; 107 (01) 25-29
    CrossrefPubMedGoogle Scholar
  • 25 Gallivan RP, Nguyen TH, Armstrong WB. Head and neck computed tomography virtual endoscopy: evaluation of a new imaging technique. Laryngoscope 1999; 109 (10) 1570-1579
    CrossrefPubMedGoogle Scholar
  • 26 Tolsdorff B, Pommert A, Höhne KH. et al. Virtual reality: a new paranasal sinus surgery simulator. Laryngoscope 2010; 120 (02) 420-426
    CrossrefPubMedGoogle Scholar
  • 27 Esteva A, Robicquet A, Ramsundar B. et al. A guide to deep learning in healthcare. Nat Med 2019; 25 (01) 24-29
    CrossrefPubMedGoogle Scholar
  • 28 Topol EJ. High-performance medicine: the convergence of human and artificial intelligence. Nat Med 2019; 25 (01) 44-56
    CrossrefPubMedGoogle Scholar
  • 29 He J, Baxter SL, Xu J, Xu J, Zhou X, Zhang K. The practical implementation of artificial intelligence technologies in medicine. Nat Med 2019; 25 (01) 30-36
    CrossrefPubMedGoogle Scholar
  • 30 Panch T, Szolovits P, Atun R. Artificial intelligence, machine learning and health systems. J Glob Health 2018; 8 (02) 020303
    CrossrefPubMedGoogle Scholar
  • 31 Humphries SM, Centeno JP, Notary AM. et al. Volumetric assessment of paranasal sinus opacification on computed tomography can be automated using a convolutional neural network. Int Forum Allergy Rhinol 2020; 10 (11) 1218-1225
    CrossrefPubMedGoogle Scholar
  • 32 Iwamoto Y, Xiong K, Kitamura T. et al. Automatic segmentation of the paranasal sinus from computer tomography images using a probabilistic atlas and a fully convolutional network. Annu Int Conf IEEE Eng Med Biol Soc 2019; 2019: 2789-2792
    PubMedGoogle Scholar
  • 33 Souadih K, Belaid A, Ben Salem D, Conze PH. Automatic forensic identification using 3D sphenoid sinus segmentation and deep characterization. Med Biol Eng Comput 2020; 58 (02) 291-306
    CrossrefPubMedGoogle Scholar
  • 34 Seo H, Badiei Khuzani M, Vasudevan V. et al. Machine learning techniques for biomedical image segmentation: an overview of technical aspects and introduction to state-of-art applications. Med Phys 2020; 47 (05) e148-e167
    PubMedGoogle Scholar
  • 35 Kuo C-CJ, Chen Y. On data-driven Saak transform. J Vis Commun Image Represent 2018; 50: 237-246
    CrossrefPubMedGoogle Scholar
  • 36 Ding Y, Gudapati V, Lin R. et al. Saak transform-based machine learning for light-sheet imaging of cardiac trabeculation. IEEE Trans Biomed Eng 2021; 68 (01) 225-235
    CrossrefPubMedGoogle Scholar
  • 37 Weber RK, Hosemann W. Comprehensive review on endonasal endoscopic sinus surgery. GMS Curr Top Otorhinolaryngol Head Neck Surg 2015; 14: Doc08
    PubMedGoogle Scholar