Best Paper Selection

Banerjee I, Gensheimer MF, Wood DJ, Henry S, Aggarwal S, Chang DT, Rubin DL. Probabilistic prognostic estimates of survival in metastatic cancer patients (PPES-Met) utilizing free-text clinical narratives. Sci Rep 2018 Jul 3;8(1):10037 https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6030075/

Ray S, McEvoy DS, Aaron S, Hickman TT, Wright A. Using statistical anomaly detection models to find clinical decision support malfunctions. J Am Med Inform Assoc 2018 Jul 1;25(7):862-71 https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6016695/

Simon G, DiNardo CD, Takahashi K, Cascone T, Powers C, Stevens R, Allen J, Antonoff MB, Gomez D, Keane P, Suarez Saiz F, Nguyen Q, Roarty E, Pierce S, Zhang J, Hardeman Barnhill E, Lakhani K, Shaw K, Smith B, Swisher S, High R, Futreal PA, Heymach, Chin L. Applying Artificial Intelligence to address the knowledge gaps in cancer care. Oncologist 2018 Nov 16 pii: theoncologist.2018-0257 http://theoncologist.alphamedpress.org/content/24/6/772.long

Tsopra R, Lamy JB, Sedki K. Using preference learning for detecting inconsistencies in clinical practice guidelines: methods and application to antibiotherapy. Artif Intell Med 2018 Jul;89:24-33 https://www.sciencedirect.com/science/article/pii/S0933365718300873?via%3Dihub

Appendix: Content Summaries of Best Papers for the Decision Support Section of the 2019 IMIA Yearbook

Banerjee I, Gensheimer MF, Wood DJ, Henry S, Aggarwal S, Chang DT, Rubin DL

Probabilistic prognostic estimates of survival in metastatic cancer patients (PPES-Met) utilizing free-text clinical narratives

Sci Rep 2018 Jul 3;8(1):10037

Following a deep learning approach for the analysis of free-text clinical notes originated from various health IT modalities, the study focused on the probabilistic prognostic estimation of survival in metastatic cancer patients by proposing the so-called PPES-Met model. The work extends prior works targeting the prediction of complex disease trajectories by exploiting rich unstructured clinical data, rather than structured (e.g. lab values, demographics, etc.), or relatively simplistic information extracted from unstructured narratives (e.g. bag of words and term frequency-inverse document frequency). In technical terms, PPES-Met combines semantic data mining and neural embedding for creating a context-aware dense vector representation of the clinical notes, which was used as input to the prognostic model for estimating in turn the probability of short-term life expectancy (>3 months). The model was trained on a large dataset (10,293 patients) and validated on a separated dataset (1,818 patients), exhibiting an AUC (area under the ROC curve) value of 0.89. Equally important, aiming to facilitate the explain- ability of the prediction, PPES-Met offers an interactive visualization method for the end-user. Overall, the study introduced a data-driven approach for a promising decision support tool targeting personalized metastatic cancer treatment.

Ray S, McEvoy DS, Aaron S, Hickman TT, Wright A

Using statistical anomaly detection models to find clinical decision support malfunctions

J Am Med Inform Assoc 2018 Jul 1;25(7):862-71

Taking note that CDSSs become routinely integrated in health information systems and electronic health records (EHRs), the authors stress that malfunctions of these systems, if unnoticed, may have a negative impact on care delivery. The objective of this work was to use and compare anomaly detection models to see to what extent they could identify CDS malfunctions. The focus was on anomalies in time series, when there is a change or unexpected variation, more specifically on change-point anomaly, mean-shift anomaly, and mean-drift anomaly. Six statistical anomaly detection models were compared on retrospective data from EHRs at Brigham and Women’s Hospital, Boston, MA, on four CDS alerts, implementing four clinical rules, with known malfunctions. Malfunctions were due to changes in the code of the CDS, or changes in terminology codes of the information system. They led alerts to stop firing, or to fire for more patients than relevant. The six models performed differently for each type of anomaly. These models were able to detect anomalies with offline data. Perspectively, they might enable to find the root causes of malfunctions, but the further challenge will be to detect anomalies online, i.e., in real time as they appear.

Simon G, DiNardo CD, Takahashi K, Cascone T, Powers C, Stevens R, Allen J, Antonoff MB, Gomez D, Keane P, Suarez Saiz F, Nguyen Q, Roarty E, Pierce S, Zhang J, Hardeman Barnhill E, Lakhani K, Shaw K, Smith B, Swisher S, High R, Futreal PA, Heymach, Chin L

Applying Artificial Intelligence to address the knowledge gaps in cancer care

Oncologist 2018 Nov 16 pii: theoncologist.2018-0257

This article reports on lessons learnt from the development and the introduction of a large, AI-powered CDSS, in a large US cancer center. The aim was to bridge the gap between what is practiced and what is possible by promoting evidence-based care in real time. The CDSS, called the Oncology Expert Advisor (OEA), was built using IBM Watson’s technologies. OEA provided three clinical support functions: patient history summarization from EHR data and documents, recommendation of treatment options and clinical trials, and management advisory. It was first applied to leukemia, then to lung cancer. Retrospective data from around 1,000 patients were used to train machine-learning algorithms. Patient summarization through mining patient records performed with good results for non time-dependent concepts, but was less efficient for time-dependent concepts. Suggestion of therapy options with links to supporting evidence had good recall and precision (99.9% and 88%, respectively). A controlled introduction was performed in a clinic team, where errors were collected and analyzed. From their experience, authors conclude that AI-based approaches to decision support are technically feasible, but clinical expertise should be taken into account earlier and more extensively in the development process of such CDS applications.

Tsopra R, Lamy JB, Sedki K

Using preference learning for detecting inconsistencies in clinical practice guidelines: methods and application to antibiotherapy

Artif Intell Med 2018 Jul;89:24-33

Contradictions and inconsistencies in clinical practice guidelines (CPGs) are well-known problems in the domain of CPG-based CDSSs. This study employed preference learning to develop a method for the semi-automatic detection of inconsistencies in CPGs. The application focus of the study was antibiotherapy in the primary care setting. Key elements of the proposed approach included the adoption of the Artificial Feeding Birds (AFB) metaheuristic as a learning optimization algorithm and a knowledge base of the domain. The knowledge base was built and populated by a medical doctor through a two step-process, incorporating information related to 11 infectious diseases, the 50 antibiotics marketed for use in primary care in France, and 21 patient profiles. The knowledge base associates infectious diseases with the likely causative bacteria, describes a patient profile by the age class and the presence or absence of pregnancy, allergy, and history of antibiotic treatment, and corresponds a clinical situation to the intersection of an infectious disease and a patient profile. Interestingly, preference model learning relied both on CPG recommendations and on the knowledge base. Preference learning on the antibiotic knowledge base allowed the detection of 106 errors by a medical expert, 55 of which originated from CPG inconsistencies, 17 from flaws in the antibiotic knowledge base, 16 from by flaws in the preference model, while the rest could not be categorized by the medical expert. The authors argue that they successfully built a generic model suitable for all infectious diseases and patient profiles, offering a comprehensive CDSS for antibiotics prescription.