PDF herunterladen
CC BY 4.0 · Yearb Med Inform 2024; 33(01): 018-024
DOI: 10.1055/s-0044-1800714
Special Section: Digital Health for Precision in Prevention
Working Group Contributions

Safety and Precision AI for a Modern Digital Health System

Elizabeth M. Borycki
1   School of Health Information Science, University of Victoria, Victoria, British Columbia, Canada
,
Linda W. P. Peute
2   Department of Medical Informatics, Amsterdam UMC Location, University of Amsterdam Amsterdam, The Netherlands
,
Femke van Sinderen
2   Department of Medical Informatics, Amsterdam UMC Location, University of Amsterdam Amsterdam, The Netherlands
,
David Kaufman
3   School of Health Professions Faculty, Health/Medical Informatics, SUNY Downstate Health Sciences University, Brooklyn, NY
,
Andre W. Kushniruk
1   School of Health Information Science, University of Victoria, Victoria, British Columbia, Canada
› Institutsangaben
› Weitere Informationen
Auch verfügbar auf
   Lizenzen und Reprints

Summary

Artificial intelligence (AI) promises to revolutionize healthcare. Currently there is a proliferation of new AI applications that are being developed and beginning to be deployed across many areas in healthcare to streamline and make healthcare processes more efficient. In addition, AI has the potential to support personalized and customized precision healthcare by providing intelligent interaction with end users. However, to achieve the goal of precision AI issues and concerns related to the safety of AI, as with any new technology, must be addressed. In this article we first describe the link between AI and safety and then describe the relation of AI to the emerging study of technology-induced error. An overview of published safety issues that have been associated with introduction of AI are described and categorized. These include potential for error to arise from varied sources, including the data used to drive AI applications, and the design process of AI applications itself. In addition, lack of appropriate and rigorous testing and limited analysis of AI applications during procurement processes has also been reported. Recommendations for ensuring the safe adoption of AI technology in healthcare are discussed, focusing on the need for more rigorous testing and evaluation of AI applications, ranging from laboratory testing through to naturalistic evaluation. The application of such approaches will support safety and precision AI for a modern digital health system.



Publikationsverlauf

Artikel online veröffentlicht:
08. April 2025

© 2024. The Author(s). This is an open access article published by Thieme under the terms of the Creative Commons Attribution License, permitting unrestricted use, distribution, and reproduction so long as the original work is properly cited. (https://creativecommons.org/licenses/by/4.0/)

Georg Thieme Verlag KG
Rüdigerstraße 14, 70469 Stuttgart, Germany

 

  • References

  • 1 Choudhury A, Asan O. (2020). Role of artificial intelligence in patient safety outcomes: Systematic literature review. JMIR Med Inform 2020;8(7):e18599
    PubMed
  • 2 Cohen TA, Patel VL, Shortliffe EH, editors. Intelligent Systems in Medicine and Health: The Role of AI. Springer Nature; 2022 Nov 9.
    PubMed
  • 3 Househ M., Borycki E., Kushniruk A. Multiple Perspectives on Artificial Intelligence in Healthcare Opportunities and Challenges: Opportunities and Challenges. Springer Verlag. 2021 DOI:10.1007/978-3-030-67303-1 ISBN: 978-3-030-67302-4.
    PubMed
  • 4 Leslie D. Understanding Artificial Intelligence Ethics and Safety: A Guide for Responsible Design and Implementation of AI Systems in the Public Sector. The Alan Turing Institute: London, England. 2019.
    PubMed
  • 5 Borycki EM, Kushniruk AW. The safety of AI in healthcare: Emerging issues and considerations for Healthcare. In Multiple Perspectives on Artificial Intelligence in Healthcare, Househ, M., Borycki, E and Kushniruk A. (Eds.). Springer Verlag: New York. 2021.
    PubMed
  • 6 Yampolskiy RV. Artificial intelligence safety and security. Taylor and Francis Group, 2019.
    PubMed
  • 7 Borycki EM, Kushniruk AW. Health technology, quality and safety in a learning health system. Healthc Manage Forum. 2023 Mar;36(2):79-85. doi: 10.1177/08404704221139383. Epub 2022 Dec 23. PMID: 36562483; PMCID: PMC9975897.
    PubMed
  • 8 Frackiewicz M. The impact of Elon Musk's OpenAI on AI-enabled healthcare innovation. Artificial Intelligence 2023 Apr https://ts2.pl/en/the-impact-of-elon-musks-openai-on-ai-enabled-healthcare-innovation/#gsc.tab=0CRC Press.
    PubMed
  • 9 Landymore F. OpenAI CEO says AI will give medical advice to people too poor to afford doctors: An “AI medical advisor”? What could go wrong? The_Byte. https://futurism.com/the-byte/openai-ceo-ai-medical-advice
    PubMed
  • 10 Chen ZH, Lin L, Wu CF, Li CF, Xu RH, Sun Y. Artificial intelligence for assisting cancer diagnosis and treatment in the era of precision medicine. Cancer Commun (Lond). 2021 Nov;41(11):1100-1115. Doi: 10.1002/cac2.12215. Epub 2021 Oct 6. PMID: 34613667; PMCID: PMC8626610.
    PubMed
  • 11 Henshall W. AI experts call for policy action to avoid extreme risks. Time, Oct 24, 2023 https://time.com/6328111/open-letter-ai-policy-action-avoid-extreme-risks/
    PubMed
  • 12 Grant MJ, Booth A. A typology of reviews: an analysis of 14 review types and associated methodologies. Health Information & Libraries Journal 2009; 26, 91–108. doi:10.1111/j.1471-1842.2009.00848.x
    PubMed
  • 13 Davy A, Borycki EM. Copy and paste in the electronic medical record: A scoping review. Knowledge Management & E-Learning: An International Journal 2021, 13(4), 522-535. https://doi.org/10.34105/j.kmel.2021.13.028
    PubMed
  • 14 Li Y, Shyr C, Borycki EM, Kushniruk AW. Automated thematic analysis of health information technology (HIT) related incident reports. Knowledge Management & E-Learning 2021, 13(4), 408–420. https://doi.org/10.34105/j.kmel.2021.13.022
    PubMed
  • 15 Xx
    PubMed
  • 16 About Learning Health Systems. Content last reviewed May 2019. Agency for Healthcare Research and Quality, Rockville, MD. https://www.ahrq.gov/learning-health-systems/about.html
    PubMed
  • 17 Ash JS, Sittig DF, Poon EG, Guappone K, Campbell E, Dykstra RH. The extent and importance of unintended consequences related to computerized provider order entry. Journal of the American Medical Informatics Association 2007, 14(4), 415-23.
    PubMed
  • 18 Ash JS, Berg M, Coiera E. Some unintended consequences of information technology in health care: the nature of patient care information system-related errors. Journal of the American Medical Informatics Association 2004, 11(2), 104-112.
    PubMed
  • 19 Kushniruk AW, Triola MM, Borycki EM, Stein B, Kannry JL. Technology induced error and usability: The relationship between usability problems and prescription errors when using a handheld application. International Journal of Medical Informatics 2005, 74(7-8), 519-526.
    PubMed
  • 20 Institute of Medicine (US) Committee on Quality of Health Care in America. To Err is Human: Building a Safer Health System. Kohn LT, Corrigan JM, Donaldson MS, editors. Washington (DC): National Academies Press (US); 2000. PMID: 25077248.
    PubMed
  • 21 Borycki EM, Kushniruk AW, Bellwood P, Brender J. Technology-induced errors: The current use of frameworks and models from the biomedical and life sciences literatures. Methods of Information in Medicine 2012, 51(02), 95-103.
    PubMed
  • 22 Khan ME, Khan F. A Comparative Study of White Box, Black Box and Grey Box Testing Techniques. International Journal of Advanced Computer Science and Applications(IJACSA), 3(6), 2012. http://dx.doi.org/10.14569/IJACSA.2012.030603
    PubMed
  • 23 Palojoki S, Pajunen T, Saranto K, Lehtonen, L. Electronic health record-related safety concerns: a cross-sectional survey of electronic health record users. JMIR medical informatics, 4(2), 2016.
    PubMed
  • 24 Palojoki S, Mäkelä M, Lehtonen L, Saranto K. An analysis of electronic health record-related patient safety incidents. Health Informatics J. 2017 Jun;23(2):134-145. doi: 10.1177/1460458216631072. Epub 2016 Mar 7.
    PubMed
  • 25 Borycki EM, Griffith J, Monkman H, Reid-Haughian C. Isolating the Effects of a Mobile Phone on the Usability and Safety of eHealth Software Applications. Stud Health Technol Inform. 2017;234:37-41. PMID: 28186012.
    PubMed
  • 26 Magrabi F, Ong MS, Runciman W, Coiera E. Using FDA reports to inform a classification for health information technology safety problems. J Am Med Inform Assoc. 2012 Jan-Feb;19(1):45-53. doi: 10.1136/amiajnl-2011-000369. Epub 2011 Sep 8. PMID: 21903979; PMCID: PMC3240763.
    PubMed
  • 27 Akbar S, Coiera E, Magrabi F. Safety concerns with consumer-facing mobile health applications and their consequences: a scoping review, Journal of the American Medical Informatics Association, 27(2), 2020, 330–340, https://doi.org/10.1093/jamia/ocz175
    PubMed
  • 28 Borycki EM, Kushniruk AW, Kuwata S, Kannry J. Engineering the electronic health record for safety: a multi-level video-based approach to diagnosing and preventing technology-induced error arising from usability problems. Stud Health Technol Inform. 2011;166:197-205. PMID: 21685625.
    PubMed
  • 29 Jensen S, Kushniruk AW, Nøhr C. Clinical simulation: A method for development and evaluation of clinical information systems. J Biomed Inform. 2015 Apr;54:65-76. doi: 10.1016/j.jbi.2015.02.002. Epub 2015 Feb 12. PMID: 25684129.
    PubMed
  • 30 Magrabi F, Baker M, Sinha I, Ong MS, Harrison S, Kidd MR, Runciman WB, Coiera E. Clinical safety of England's national programme for IT: a retrospective analysis of all reported safety events 2005 to 2011. Int J Med Inform. 2015 Mar;84(3):198-206. doi: 10.1016/j.ijmedinf.2014.12.003. Epub 2015 Jan 4. PMID: 25617015.
    PubMed
  • 31 Borycki EM, Farghali A, Kushniruk AW. Do Health Technology Safety Issues Vary by Vendor? Stud Health Technol Inform. 2022 Jun 29;295:345-349. doi: 10.3233/SHTI220734. PMID: 35773880.
    PubMed
  • 32 Cheung KC, van der Veen W, Bouvy ML, Wensing M, van den Bemt PM, de Smet PA. Classification of medication incidents associated with information technology. J Am Med Inform Assoc. 2014 Feb;21(e1):e63-70. doi: 10.1136/amiajnl-2013-001818. Epub 2013 Sep 24. PMID: 24064444; PMCID: PMC3957393.
    PubMed
  • 33 Burgess M, Kobie N. (March 18, 2019). The messy, cautionary tale of how Babylon disrupted the NHS. Wired.
    PubMed
  • 34 El Morr, C. Introduction to health informatics: A Canadian perspective. Canadian Scholars: Toronto, 2018.
    PubMed
  • 35 Kushniruk A, Beuscart-Zéphir MC, Grzes A, Borycki E, Watbled L, Kannry J. Increasing the safety of healthcare information systems through improved procurement: toward a framework for selection of safe healthcare systems. Healthc Q. 2010;13 Spec No:53-8. doi: 10.12927/hcq.2010.21967. PMID: 20959731.
    PubMed
  • 36 Stogiannos N, Malik R, Kumar A, Barnes A, Pogose M, Harvey H, et al. Black box no more: a scoping review of AI governance frameworks to guide procurement and adoption of AI in medical imaging and radiotherapy in the UK. Br J Radiol. 2023 Dec;96(1152):20221157. doi: 10.1259/bjr.20221157. Epub 2023 Oct 3. PMID: 37747285; PMCID: PMC10646619.
    PubMed
  • 37 Kannry J, Mukani S, Myers K. Using an Evidence-Based Approach for System Selection at a Large Academic Medical Center: Lessons Learned in Selecting an Ambulatory EMR at Mount Sinai Hospital. Journal of Healthcare Information Management 2006; 20(2): 99.
    PubMed
  • 38 Wunn T, Sent D, Peute LWP, Leijnen S. Trust in Artificial Intelligence: Exploring the Influence of Model Presentation and Model Interaction on Trust in a Medical Setting. Studies in Health Technology and Informatics, 2024.
    PubMed
  • 39 Goddard K, Roudsari A, Wyatt JC. Automation bias: a systematic review of frequency, effect mediators, and mitigators. J Am Med Inform Assoc. 2012 Jan-Feb;19(1):121-7. doi: 10.1136/amiajnl-2011-000089. Epub 2011 Jun 16. PMID: 21685142; PMCID: PMC3240751.
    PubMed
  • 40 Kushniruk A, Nohr C, Jensen S, Borycki EM. From usability testing to clinical simulations: Bringing context into the design and evaluation of usable and safe health information technologies. Yearbook of medical informatics 2013, 22(01), 78-85.
    PubMed