An Open Science Approach to Artificial Intelligence in Healthcare

Summary

Objectives: Artificial Intelligence (AI) offers significant potential for improving healthcare. This paper discusses how an “open science” approach to AI tool development, data sharing, education, and research can support the clinical adoption of AI systems.

Method: In response to the call for participation for the 2019 International Medical Informatics Association (IMIA) Yearbook theme issue on AI in healthcare, the IMIA Open Source Working Group conducted a rapid review of recent literature relating to open science and AI in healthcare and discussed how an open science approach could help overcome concerns about the adoption of new AI technology in healthcare settings.

Results: The recent literature reveals that open science approaches to AI system development are well established. The ecosystem of software development, data sharing, education, and research in the AI community has, in general, adopted an open science ethos that has driven much of the recent innovation and adoption of new AI techniques. However, within the healthcare domain, adoption may be inhibited by the use of “black-box” AI systems, where only the inputs and outputs of those systems are understood, and clinical effectiveness and implementation studies are missing.

Conclusions: As AI-based data analysis and clinical decision support systems begin to be implemented in healthcare systems around the world, further openness of clinical effectiveness and mechanisms of action may be required by safety-conscious healthcare policy-makers to ensure they are clinically effective in real world use.

Keywords

Artificial Intelligence - machine learning - expert systems - open science - open source

• References

• 1 Russel SJ, Norvig P. Artificial intelligence: a modern approach. Malaysia: Pearson Education Limited 2016
• 2 Armstrong S. The apps attempting to transfer NHS 111 online. BMJ 2018; 360: k156
• 3 Fraser H, Coiera E, Wong D. Safety of patient-facing digital symptom checkers. Lancet 2018; 392: 2263-4
• 4 Michalski RS, Carbonell JG, Mitchell TM. Machine Learning: An Artificial Intelligence Approach. Springer Science & Business Media 2013
• 5 Bradley AP. The use of the area under the ROC curve in the evaluation of machine learning algorithms. Pattern Recognit 1997; 30: 1145-59
• 6 Silver D, Huang A, Maddison CJ, Guez A, Sifre L, van den Driessche G. et al. Mastering the game of Go with deep neural networks and tree search. Nature 2016; 529: 484-9
• 7 Hosny A, Parmar C, Quackenbush J, Schwartz LH, Aerts HJWL. Artificial intelligence in radiology. Nat Rev Cancer 2018; 18: 500-10
• 8 Fogel AL, Kvedar JC. Artificial intelligence powers digital medicine. NPJ Digit Med 2018; 1: 5
• 9 Zakhem GA, Motosko CC, Ho RS. How Should Artificial Intelligence Screen for Skin Cancer and Deliver Diagnostic Predictions to Patients?. JAMA Dermatol 2018; 154: 1383-4
• 10 Salto-Tellez M, Maxwell P, Hamilton P. Artificial Intelligence-The Third Revolution in Pathology. Histopathology 2018 Available from: https://onlinelibrary.wiley.com/doi/abs/10.1111/his.13760
• 11 Orphanidou C, Bonnici T, Charlton P, Clifton D, Vallance D, Tarassenko L. Signal-quality indices for the electrocardiogram and photoplethysmogram: derivation and applications to wireless monitoring. IEEE J Biomed Health Inform 2015; 19: 832-8
• 12 Triantafyllidis A, Velardo C, Shah SA, Tarassenko L, Chantler T, Paton C. et al. Supporting heart failure patients through personalized mobile health monitoring. In: Wireless Mobile Communication and Healthcare (Mobihealth), 2014 EAI 4th International Conference on IEEE 2014; 287-90
• 13 Triantafyllidis A, Velardo C, Chantler T, Shah SA, Paton C, Khorshidi R. et al; SUPPORT-HF Investigators A personalised mobile-based home monitoring system for heart failure: The SUPPORT-HF Study. Int J Med Inform 2015; 84: 743-53
• 14 Pontika N, Knoth P, Cancellieri M, Pearce S. Fostering Open Science to Research Using a Taxonomy and an eLearning Portal. In: Proceedings of the 15th International Conference on Knowledge Technologies and Data-driven Business. New York, NY, USA: ACM 2015 11:1-11:8
• 15 Open Knowledge Open Definition Group. Open Definition 2.1 - Open Definition - Defining Open in Open Data, Open Content and Open Knowledge. Available from: http://opendefinition.org/od/2.1/en/
• 16 Kobayashi S, Kane TB, Paton C. The Privacy and Security Implications of Open Data in Healthcare. Yearb Med Inform 2018; 27: 41-7
• 17 Edgar R, Domrachev M, Lash AE. Gene Expression Omnibus: NCBI gene expression and hybridization array data repository. Nucleic Acids Res 2002; 30: 207-10
• 18 Benson DA, Karsch-Mizrachi I, Lipman DJ, Ostell J, Rapp BA, Wheeler DL. GenBank. Nucleic Acids Res 2000; 28: 15-8
• 19 UniProt Consortium T. UniProt: the universal protein knowledgebase. Nucleic Acids Res 2018; 46: 2699
• 20 Finn RD, Mistry J, Schuster-Böckler B, Griffiths-Jones S, Hollich V, Lassmann T. et al. Pfam: clans, web tools and services. Nucleic Acids Res 2006; 34: D247-51
• 21 Berman HM, Westbrook J, Feng Z, Gilliland G, Bhat TN, Weissig H. et al. The Protein Data Bank. Nucleic Acids Res 2000; 28: 235-42
• 22 Ashburner M, Ball CA, Blake JA, Botstein D, Butler H, Cherry JM. et al. Gene ontology: tool for the unification of biology. The Gene Ontology Consortium. Nat Genet 2000; 25: 25-9
• 23 Szklarczyk D, Franceschini A, Wyder S, Forslund K, Heller D, Huerta-Cepas J. et al. STRING v10: protein-protein interaction networks, integrated over the tree of life. Nucleic Acids Res 2015; 43: D447-52
• 24 Boyle AP, Hong EL, Hariharan M, Cheng Y, Schaub MA, Kasowski M. et al. Annotation of functional variation in personal genomes using RegulomeDB. Genome Res 2012; 22: 1790-7
• 25 Armato SG, 3rd, McLennan G, Bidaut L, McNitt-Gray MF, Meyer CR, Reeves AP. et al. The Lung Image Database Consortium (LIDC) and Image Database Resource Initiative (IDRI): a completed reference database of lung nodules on CT scans. Med Phys 2011; 38: 915-31
• 26 Irvin J, Rajpurkar P, Ko M, Yu Y, Ciurea-Ilcus S, Chute C. et al. CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison. arXiv [csCV] 2019 Available from: http://arxiv.org/abs/1901.07031
• 27 Greenhalgh T, Hinder S, Stramer K, Bratan T, Russell J. Adoption, non-adoption, and abandonment of a personal electronic health record: case study of HealthSpace. BMJ 2010; 341: c5814
• 28 Amershi S, Weld D, Vorvoreanu M, Fourney A, Nushi B, Collisson P. et al. Guidelines for Human-AI Interaction. 2019 Available from: http://dx.doi.org/10.1145/3290605.3300233
• 29 Artificial Intelligence in healthcare - Academy of Medical Royal Colleges. Academy of Medical Royal Colleges 2019 Available from: https://www.aomrc.org.uk/reports-guidance/artificial-intelligence-in-healthcare/
• 30 Evidence standards framework for digital health technologies. National Institute for Health and Care Excellence Available from: https://www.nice.org.uk/about/what-we-do/our-programmes/evidence-standards-framework-for-digital-health-technologies
• 31 Greaves F, Joshi I, Campbell M, Roberts S, Patel N, Powell J. What is an appropriate level of evidence for a digital health intervention?. Lancet 2018; 392: 2665-7
• 32 Cortez NG, Cohen IG, Kesselheim AS. FDA regulation of mobile health technologies. N Engl J Med 2014; 371: 372-9
• 33 Food and Drug Administration. Mobile medical applications: guidance for industry and Food and Drug Administration staff. FDA Med Bull 2013
• 34 Cristea IA, Cahan EM, Ioannidis JPA. Stealth research: lack of peer-reviewed evidence from healthcare unicorns. Eur J Clin Invest 2019; e13072
• 35 Statement on Nature Machine Intelligence. Oregon State University. Available from: https://openaccess.engineering.oregonstate.edu/
• 36 Year in Review: 10 Most Popular Courses in 2017. Coursera Blog 2017; Available from: https://blog.coursera.org/year-review-10-popular-courses-2017/
• 37 Paton C. Massive open online course for health informatics education. Healthc Inform Res 2014; 20: 81-7
• 38 Paton C, Malik M, Househ M. Disseminating health informatics research and Informatics in Developing Countries and the Health Informatics Forum MOOC. Australas Epidemiol 2014; 21: 29-32
• 39 Konstantinidis ST, Hansen M, Bamidis PD, Paton C. Health Education in the Era of Social Media, the Semantic Web and MOOCs. Stud Health Technol and Inform 2013(192) :MEDINFO :2013. IOS Press 2013
• 40 Koch S, Hägglund M. Mutual Learning and Exchange of Health Informatics Experiences from Around the World - Evaluation of a Massive Open Online Course in eHealth. Stud Health Technol Inform 2017; 245: 753-7
• 41 Coudray N, Ocampo PS, Sakellaropoulos T, Narula N. Classification and mutation prediction from non-small cell lung cancer histopathology images using deep learning. Nat Med 2018; 24 (010) 1559-67
• 42 TensorFlow. TensorFlow Available from: https://www.tensorflow.org/
• 43 Scientific computing for LuaJIT. Available from: http://torch.ch/
• 44 Microsoft Cognitive Toolkit. Microsoft Cognitive Toolkit Available from: https://www.microsoft.com/en-us/cognitive-toolkit/
• 45 Chainer: A flexible framework for neural networks. Chainer Available from: https://chainer.org/
• 46 Caffe2. Caffe2 Available from: https://caffe2.ai/
• 47 scikit-learn: machine learning in Python — scikit-learn 0.20.2 documentation. Available from: https://scikit-learn.org/stable/
• 48 amazon-dsstne. Github Available from: https://github.com/amzn/amazon-dsstne
• 49 Team RC. R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. 2013 2014
• 50 Frank E., Hall M.A., Witten IH. The WEKA Workbench. Morgan Kaufmann 2016
• 51 Trapnell C., Pachter L., Salzberg S.L.. TopHat: discovering splice junctions with RNA-Seq. Bioinformatics 2009; 25: 1105-11