RSS-Feed abonnieren
![](/products/assets/desktop/img/oa-logo.png)
DOI: 10.1055/s-0043-1772704
Leveraging Large Language Models (LLM) for the Plastic Surgery Resident Training: Do They Have a Role?
![](https://www.thieme-connect.de/media/10.1055-s-00042863/202305/lookinside/thumbnails/10-1055-s-0043-1772704_2362230-1.jpg)
Abstract
Introduction Large language models (LLMs) are designed for recognizing, summarizing, translating, predicting, and generating text-based content from knowledge gained from extensive data sets. ChatGPT4 (Generative Pre-trained Transformer 4) (OpenAI, San Francisco, California, United States) is a transformer-based LLM model pretrained on public data as well as data obtained from third-party sources using deep learning techniques of fine tuning and reinforcement learning from human feedback to predict the next text. We wanted to explore the role of LLM as a teaching assistant (TA) in plastic surgery.
Material and Methods TA roles were first identified in available literature, and based on the roles, a list of suitable tasks was created where LLM could be used to perform the task. Prompts designed to be fed in to the LLM (specifically ChatGPT) to generate appropriate output, were then created and fed to the ChatGPT model. The outputs generated were scored by evaluators and compared for interobserver agreement.
Results A final set of eight TA roles were identified where a LLM could be utilized to generate content. These contents were scored for usefulness and accuracy. These were scored independently by the eight study authors in a scoring sheet created for the study. Interobserver agreements for content accuracy, usefulness, and clarity were 100% for content generated for the following: interactive case studies (generation), simulation of preoperative consultations, and generation of ethical considerations.
Discussion LLMs in general and ChatGPT (on which this study is based) in specific, can generate answers to questions and prompts based on huge amount of text fed into the model for training the underlying language model. The answers generated have been found to be accurate, readable, and even indistinguishable from human-generated text. This capability of automated content synthesis can be exploited to generate summaries to text, answer short and long answers, and generate case scenarios. We could identify a few such scenarios where the LLM could in general be utilized to play the role of a TA and aid plastic surgery residents in particular. In addition, these models could also be used by students to obtain feedback and gain reflection which itself stimulates critical thinking.
Conclusion Incorporating LLMs into the educational arsenal of plastic surgery residency programs can provide a dynamic, interactive, and individualized learning experience for residents and prove to be worthy TAs of future.
Keywords
large language models (LLM) - plastic surgical education - educational technology - future of surgical training - ChatGPT in educationPublikationsverlauf
Artikel online veröffentlicht:
28. August 2023
© 2023. Association of Plastic Surgeons of India. This is an open access article published by Thieme under the terms of the Creative Commons Attribution-NonDerivative-NonCommercial License, permitting copying and reproduction so long as the original work is given appropriate credit. Contents may not be used for commercial purposes, or adapted, remixed, transformed or built upon. (https://creativecommons.org/licenses/by-nc-nd/4.0/)
Thieme Medical and Scientific Publishers Pvt. Ltd.
A-12, 2nd Floor, Sector 2, Noida-201301 UP, India
-
References
- 1 Rouse M. Large Language Model (LLM). April 28, 2023 https://www.techopedia.com/ . Accessed on May 5, 2023 at: https://www.techopedia.com/definition/34948/large-language-model-llm#:~:text=level%20of%20accuracy.-,How%20Do%20Large%20Language%20Models%20Work%3F,implementation%20of%20a%20transformer%20architecture
- 2 Open AI. ChatGPT. Zugriff am 08. Februar 2023 unter: https://openai.com/blog/chatgpt
- 3 Open AI. GPT-4 technical report. 2023. arXiv. Zugriff am 07. August 2023 unter: https://arxiv.org/abs/2303.08774
- 4 Wolfram S. What is ChatGPT doing ... and why does it work?. Stephen Wolfram Writings. February 14, 2023 Zugriff am 07. August 2023 unter: https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-doing-and-why-does-it-work/
- 5 Riedl M. A Very Gentle Introduction to Large Language Models Without the Hype. Medium. April 14, 2023 Zugriff am 07. August 2023 unter: https://mark-riedl.medium.com/a-very-gentle-introduction-to-large-language-models-without-the-hype-5f67941fa59e
- 6 Li J, Dada A, Kleesiek J, Egger J. ChatGPT in Healthcare: A Taxonomy and Systematic Review. MedRxiv. 2023 Zugriff am 04. April 2023 unter: https://doi.org/10.1101/2023.03.30.23287899
- 7 University of Wisconsin-Milwaukee. ( n.d. ). Roles and Responsibilities of Teaching Assistants. Graduate Assistants. Zugriff am 20. April 2023 unter: https://uwm.edu/graduate-assistants/handbook/teaching-assistants/roles-and-responsibilities-of-teaching-assistants/
- 8 Randolph JJ. Online Kappa Calculator [Computer software]. 2008. Zugriff am 07. August 2023 unter: http://justusrandolph.net/kappa/
- 9 Korinek A. Exploring the impact of language models on cognitive automation with David Autor, ChatGPT, and Claude. Brookings. March 6, 2023 ; Zugriff am 22. März 2023 unter: https://www.brookings.edu/research/exploring-the-impact-of-language-models/#:~:text=One%20of%20the%20key%20advantages,more%20accurate%20and%20appropriate%20responses
- 10 Johnson D, Goodman R, Patrinely J. et al. Assessing the accuracy and reliability of AI-generated medical responses: an evaluation of the Chat-GPT model. Res Square 2023. rs.3.rs-2566942. https://doi.org/10.21203/rs.3.rs-2566942/v1 PubMed
- 11 Milano S, McGrane JA, Leonelli S. Large language models challenge the future of higher education. Nat Mach Intell 2023; 5: 333-334
- 12 Bagadood NH, Saigh BH. Teaching assistants as a prerequisite for best practice in special education settings in Saudi Arabia. Int J Comput Sci Network Security 2022; 22 (03) 101-1C06
- 13 Lachman N, Christensen KN, Pawlina W. Anatomy teaching assistants: facilitating teaching skills for medical students through apprenticeship and mentoring. Med Teach 2013; 35 (01) e919 e925.
- 14 Wiggers K. The emerging types of language models and why they matter. TechCrunch. April 28, 2022 ; Zugriff am 07. August 2023 unter: https://techcrunch.com/2022/04/28/the-emerging-types-of-language-models-and-why-they-matter/
- 15 Cheng K, Li Z, Li C. et al. The potential of GPT-4 as an AI-powered virtual assistant for surgeons specialized in joint arthroplasty. Ann Biomed Eng 2023; 51 (07) 1366-1370
- 16 Mohapatra D, Mohapatra M, Chittoria R, Friji M, Kumar S. The scope of mobile devices in health care and medical education. Int J Adv Med Health Res 2015; 2 (01) 3-8
- 17 Poda M. Large language models: the basics and their applications. Moveworks. February 9, 2023. Zugriff am 07. August 2023 unter: https://www.moveworks.com/insights/large-language-models-strengths-and-weaknesses
- 18 Maastricht University. . (n.d. ). Large Language Models and Education. Zugriff am 02. Mai 2023 unter: https://www.maastrichtuniversity.nl/large-language-models-and-education#risk
- 19 ChatGPT and Artificial Intelligence in Higher Education: Quick Start Guide [Internet]. United Nations Educational, Scientific and Cultural Organization; 2023 , Zugriff am 07. August 2023 unter: https://www.iesalc.unesco.org/wp-content/uploads/2023/04/ChatGPT-and-Artificial-Intelligence-in-higher-education-Quick-Start-guide_EN_FINAL.pdf
- 20 Tajik E, Tajik F. A comprehensive examination of the potential application of Chat GPT in higher education institutions. (Version 1). TechRxiv. 2023. Zugriff am 07. August 2023 unter: https://doi.org/10.2196/45312