The Role of ChatGPT in Microsurgery: Assessing Content Quality and Potential Applications

Dear Sir,

Artificial intelligence (AI), simply defined as intelligence demonstrated by machines, has rapidly captured the imaginations of many innovators including plastic surgeons.[1] In recent months, OpenAI's Chat Generative Pre-training Transformer (ChatGPT) has taken the center stage of both AI and mainstream discussion. ChatGPT, an example of generative AI, creates human-like text responses to virtually any prompt or follow-up question.[2] Unsurprisingly, this has been met with a great deal of both hype and controversy. Here, we present an analysis of the quality of ChatGPT-generated microsurgery content, as well as general applications and pitfalls of ChatGPT for microsurgeons.

ChatGPT has potential to streamline the clinical workflow. As plastic surgery is a field with a uniquely high proportion of elective procedures, microsurgeons may spend significant time in the process of shared decision-making. ChatGPT can improve provider efficiency by developing easy-to-understand educational resources for patients and answering associated follow-up questions. A sample frequently-asked question (FAQ) sheet generated by ChatGPT summarizing breast reconstruction with free tissue transfer is shown in [Fig. 1], covering subjects such as benefits, recovery, and complications.

Fig. 1 Breast reconstruction with free tissue transfer frequently-asked question (FAQ) sheet generated entirely by Chat Generative Pre-training Transformer (ChatGPT).

To assess the quality of patient-centered microsurgery content generated by ChatGPT, we asked ChatGPT to generate FAQ sheets for each “Common Microsurgical Procedure” listed by the American Society of Plastic Surgeons Web site. We then verified each question and answer on the FAQ sheet for accuracy and specificity. Questions with erroneous answers as compared with the current body of literature were counted as inaccurate, and answers that were not specific to the microsurgical procedure were counted as nonspecific. By these definitions, we found an average accuracy of 95.6% (standard deviation [SD] = 5.3%) and specificity of 67.8% (SD = 6.7%) for ChatGPT-generated microsurgery materials ([Fig. 2]).

Fig. 2 Accuracy and specificity of Chat Generative Pre-training Transformer (ChatGPT) frequently-asked question (FAQ) sheets generated for common microsurgical procedures.

Inaccurate answers were semantic errors rather than frank confabulations. In addition, low specificity was attributed to ChatGPT's tendency to recycle answers. For instance, ChatGPT generated accurate but identical “recovery process” answers as in [Fig. 1], regardless of nuances for particular procedures such as drain placement. In our assessment, these shortcomings are inconvenient but would not lead to medical decision-making errors.

Beyond creating educational materials for patients, ChatGPT has utility in the academic setting. Academic microsurgeons can leverage ChatGPT to assist their research, by rapidly generating first drafts, providing ideas, and troubleshooting technical issues. However, this is a controversial subject as concern for academic plagiarism, errors and bias in AI writing and lack of transparency have rapidly emerged.[2] [3] Additionally, ChatGPT cannot seek new information and therefore has a limited fund of knowledge. When commenting on uncertain subject matters, ChatGPT can provide convincing yet erroneous answers.[4] Despite these shortcomings, ChatGPT is a technology in its nascency that continues to improve.

In summary, ChatGPT and similar generative AI have potential to revolutionize health care and the field of microsurgery. We found that for microsurgical procedures, ChatGPT-generated FAQ sheets tend to be highly accurate but somewhat nonspecific. At this stage, microsurgeons should addend in more specific information while verifying general accuracy. Overall, it remains to be seen how microsurgeons respond to this exciting new development, to best incorporate ChatGPT in an advantageous yet ethical manner.

Publication History

Accepted Manuscript online:
24 May 2023

Article published online:
05 July 2023

• References

• 1 Liang X, Yang X, Yin S. et al. Artificial intelligence in plastic surgery: applications and challenges. Aesthetic Plast Surg 2021; 45 (02) 784-790
• 2 Cotton DRE, Cotton PA, Shipway JR. Chatting and Cheating. Ensuring academic integrity in the era of ChatGPT. . Published online 2023 DOI: 10.35542/osf.io/mrz8h
• 3 Biswas S. ChatGPT and the future of medical writing. Radiology 2023; 307 (02) e223312
• 4 Open AI. CHATGPT: Optimizing language models for dialogue. OpenAI. Published February 2, 2023. Accessed June 6, 2023 at: https://openai.com/blog/chatgpt/