Download PDF
Appl Clin Inform 2021; 12(01): 017-026
DOI: 10.1055/s-0040-1721010
Research Article

Leveraging Real-World Data for the Selection of Relevant Eligibility Criteria for the Implementation of Electronic Recruitment Support in Clinical Trials

Georg Melzer
1   Chair of Medical Informatics, Friedrich-Alexander University Erlangen-Nürnberg, Erlangen, Germany
,
Tim Maiwald
2   Institute for Electronics Engineering, Department Electrical Engineering, Friedrich-Alexander University Erlangen-Nürnberg, Erlangen, Germany
,
Hans-Ulrich Prokosch
1   Chair of Medical Informatics, Friedrich-Alexander University Erlangen-Nürnberg, Erlangen, Germany
,
Thomas Ganslandt
1   Chair of Medical Informatics, Friedrich-Alexander University Erlangen-Nürnberg, Erlangen, Germany
3   Heinrich-Lanz-Center for Digital Health, Department of Biomedical Informatics, Mannheim University Medicine, Ruprecht-Karls-University Heidelberg, Mannheim, Germany
› Author Affiliations Funding This study was funded in part by the European Commission within the EU/IMI project Electronic Health Records for Clinical Research (EHR4CR), grant no.: 115189.
› Further Information
Also available at
   Permissions and Reprints

Abstract

Background Even though clinical trials are indispensable for medical research, they are frequently impaired by delayed or incomplete patient recruitment, resulting in cost overruns or aborted studies. Study protocols based on real-world data with precisely expressed eligibility criteria and realistic cohort estimations are crucial for successful study execution. The increasing availability of routine clinical data in electronic health records (EHRs) provides the opportunity to also support patient recruitment during the prescreening phase. While solutions for electronic recruitment support have been published, to our knowledge, no method for the prioritization of eligibility criteria in this context has been explored.

Methods In the context of the Electronic Health Records for Clinical Research (EHR4CR) project, we examined the eligibility criteria of the KATHERINE trial. Criteria were extracted from the study protocol, deduplicated, and decomposed. A paper chart review and data warehouse query were executed to retrieve clinical data for the resulting set of simplified criteria separately from both sources. Criteria were scored according to disease specificity, data availability, and discriminatory power based on their content and the clinical dataset.

Results The study protocol contained 35 eligibility criteria, which after simplification yielded 70 atomic criteria. For a cohort of 106 patients with breast cancer and neoadjuvant treatment, 47.9% of data elements were captured through paper chart review, with the data warehouse query yielding 26.9% of data elements. Score application resulted in a prioritized subset of 17 criteria, which yielded a sensitivity of 1.00 and specificity 0.57 on EHR data (paper charts, 1.00 and 0.80) compared with actual recruitment in the trial.

Conclusion It is possible to prioritize clinical trial eligibility criteria based on real-world data to optimize prescreening of patients on a selected subset of relevant and available criteria and reduce implementation efforts for recruitment support. The performance could be further improved by increasing EHR data coverage.

Keywords

electronic health records and systems - data warehousing and data marts - secondary use - clinical trial - recruitment

Note

The present work was performed in fulfillment of the requirements for obtaining the degree “Dr. med.” from the Friedrich-Alexander University Erlangen-Nürnberg (FAU).


Protection of Human and Animal Subjects

The project was performed in compliance with the World Medical Association Declaration of Helsinki on Ethical Principles for Medical Research Involving Human Subjects, and was reviewed by the ethics board of the Medical Faculty of the University of Erlangen Nuremberg (247_14Bc).


Supplementary Material



Publication History

Received: 18 May 2020

Accepted: 04 October 2020

Article published online:
13 January 2021

© 2021. Thieme. All rights reserved.

Georg Thieme Verlag KG
Rüdigerstraße 14, 70469 Stuttgart, Germany

 

  • References

  • 1 Victor N. Registration of Clinical Studies from View of Ethics Committees (german language). Deutsches Ärzteblatt 2004; 101 (30) A-2111/B-1763/C-1695
    PubMedSearch in Google Scholar
  • 2 Cuggia M, Besana P, Glasspool D. Comparing semi-automatic systems for recruitment of patients to clinical trials. Int J Med Inform 2011; 80 (06) 371-388
    CrossrefPubMedSearch in Google Scholar
  • 3 Schumacher M, Schulgen G. Controlled clinical trials - an introduction (german language). In: Methodology of clinical studies. 2008: 1-19
    Search in Google Scholar
  • 4 Kalra D, Schmidt A, Potts H, Dupont D, Sundgren M, de Moor G. Case report from the EHR4CR project—A European Survey on Electronic Health Records Systems for Clinical Research. iHealth Connections 2011; 108-113
    PubMedSearch in Google Scholar
  • 5 Prescott RJ, Counsell CE, Gillespie WJ. et al. Factors that limit the quality, number and progress of randomised controlled trials. 1999; 3 (20) 1-143
    PubMedSearch in Google Scholar
  • 6 Fink T, Wicke D. Clinical trial - challenge with significant impact: (german language). Biotechnologie '10, '11 - Kapital, Markt, Wirtschaft. 2010. Accessed November 22, 2014
    PubMed
  • 7 EFPIA. The Pharmaceutical Industry in Figures: Key Data 2018. Accessed October 21, 2020 at: https://www.efpia.eu/media/361960/efpia-pharmafigures2018_v07-hq.pdf
    PubMed
  • 8 Getz KA, Stergiopoulos S, Short M. et al. The impact of protocol amendments on clinical trial performance and cost. Ther Innov Regul Sci 2016; 50 (04) 436-441
    CrossrefPubMedSearch in Google Scholar
  • 9 Liu K, Acharya A, Alai S, Schleyer TK. Using electronic dental record data for research: a data-mapping study. J Dent Res 2013; 92 (7, suppl) 90S-96S
    CrossrefPubMedSearch in Google Scholar
  • 10 Embi PJ, Jain A, Clark J, Harris CM. Development of an electronic health record-based clinical trial alert system to enhance recruitment at the point of care. AMIA Annu Symp Proc 2005; 2005: 231-235
    PubMedSearch in Google Scholar
  • 11 Köpcke F, Kraus S, Scholler A. et al. Secondary use of routinely collected patient data in a clinical trial: an evaluation of the effects on patient recruitment and data acquisition. Int J Med Inform 2013; 82 (03) 185-192
    CrossrefPubMedSearch in Google Scholar
  • 12 De Moor G, Sundgren M, Kalra D. et al. Using electronic health records for clinical research: the case of the EHR4CR project. J Biomed Inform 2015; 53: 162-173
    CrossrefPubMedSearch in Google Scholar
  • 13 Bruland P, Forster C, Breil B, Ständer S, Dugas M, Fritz F. Does single-source create an added value? Evaluating the impact of introducing x4T into the clinical routine on workflow modifications, data quality and cost-benefit. Int J Med Inform 2014; 83 (12) 915-928
    CrossrefPubMedSearch in Google Scholar
  • 14 Dugas M, Lange M, Müller-Tidow C, Kirchhof P, Prokosch H-U. Routine data from hospital information systems can support patient recruitment for clinical studies. Clin Trials 2010; 7 (02) 183-189
    CrossrefPubMedSearch in Google Scholar
  • 15 Bruland P, McGilchrist M, Zapletal E. et al. Common data elements for secondary use of electronic health record data for clinical trial execution and serious adverse event reporting. BMC Med Res Methodol 2016; 16 (01) 159
    CrossrefPubMedSearch in Google Scholar
  • 16 Hersh WR, Weiner MG, Embi PJ. et al. Caveats for the use of operational electronic health record data in comparative effectiveness research. Med Care 2013; 51 (08) (Suppl. 03) S30-S37
    CrossrefPubMedSearch in Google Scholar
  • 17 Weiner MG, Embi PJ. Toward reuse of clinical data for research and quality improvement: the end of the beginning?. Ann Intern Med 2009; 151 (05) 359-360
    CrossrefPubMedSearch in Google Scholar
  • 18 Weng C, Tu SW, Sim I, Richesson R. Formal representation of eligibility criteria: a literature review. J Biomed Inform 2010; 43 (03) 451-467
    CrossrefPubMedSearch in Google Scholar
  • 19 Blaisure J, Ceusters W. Business rules to improve secondary data use of electronic healthcare systems. Stud Health Technol Inform 2017; 235: 303-307
    PubMedSearch in Google Scholar
  • 20 Bache R, Taweel A, Miles S, Delaney BC. An eligibility criteria query language for heterogeneous data warehouses. Methods Inf Med 2015; 54 (01) 41-44
    Thieme ConnectPubMedSearch in Google Scholar
  • 21 Ash JS, Anderson NR, Tarczy-Hornoch P. People and organizational issues in research systems implementation. J Am Med Inform Assoc 2008; 15 (03) 283-289
    CrossrefPubMedSearch in Google Scholar
  • 22 Ateya MB, Delaney BC, Speedie SM. The value of structured data elements from electronic health records for identifying subjects for primary care clinical trials. BMC Med Inform Decis Mak 2016; 16: 1
    CrossrefPubMedSearch in Google Scholar
  • 23 Köpcke F, Trinczek B, Majeed RW. et al. Evaluation of data completeness in the electronic health record for the purpose of patient recruitment into clinical trials: a retrospective analysis of element presence. BMC Med Inform Decis Mak 2013; 13: 37
    CrossrefPubMedSearch in Google Scholar
  • 24 Doods J, Lafitte C, Ulliac-Sagnes N. et al. A European inventory of data elements for patient recruitment. Stud Health Technol Inform 2015; 210: 506-510
    PubMedSearch in Google Scholar
  • 25 Doods J, Botteri F, Dugas M, Fritz F. EHR4CR WP7. A European inventory of common electronic health record data elements for clinical trial feasibility. Trials 2014; 15: 18
    CrossrefPubMedSearch in Google Scholar
  • 26 Löbe M, Stäubert S, Goldberg C, Haffner I, Winter A. Towards phenotyping of clinical trial eligibility criteria. Stud Health Technol Inform 2018; 248: 293-299
    PubMedSearch in Google Scholar
  • 27 Wang AY, Lancaster WJ, Wyatt MC, Rasmussen LV, Fort DG, Cimino JJ. Classifying clinical trial eligibility criteria to facilitate phased cohort identification using clinical data repositories. AMIA Annu Symp Proc 2018; 2017: 1754-1763
    PubMedSearch in Google Scholar
  • 28 Weng C. Optimizing clinical research participant selection with informatics. Trends Pharmacol Sci 2015; 36 (11) 706-709
    CrossrefPubMedSearch in Google Scholar
  • 29 Rubin DL, Gennari J, Musen MA. Knowledge representation and tool support for critiquing clinical trial protocols. Proc AMIA Symp 2000; 724-728
    PubMedSearch in Google Scholar
  • 30 Ross J, Tu S, Carini S, Sim I. Analysis of eligibility criteria complexity in clinical trials. Summit On Translat Bioinforma 2010; 2010: 46-50
    PubMedSearch in Google Scholar
  • 31 Girardeau Y, Doods J, Zapletal E. et al. Leveraging the EHR4CR platform to support patient inclusion in academic studies: challenges and lessons learned. BMC Med Res Methodol 2017; 17 (01) 36
    CrossrefPubMedSearch in Google Scholar
  • 32 Trinczek B, Köpcke F, Leusch T. et al. Design and multicentric implementation of a generic software architecture for patient recruitment systems re-using existing HIS tools and routine patient data. Appl Clin Inform 2014; 5 (01) 264-283
    Thieme ConnectPubMedSearch in Google Scholar
  • 33 Murphy SN, Weber G, Mendis M. et al. Serving the enterprise and beyond with informatics for integrating biology and the bedside (i2b2). J Am Med Inform Assoc 2010; 17 (02) 124-130
    CrossrefPubMedSearch in Google Scholar
  • 34 Schreiweis B, Bergh B. Requirements for a patient recruitment system. Stud Health Technol Inform 2015; 210: 521-525
    PubMedSearch in Google Scholar
  • 35 Van Spall HGC, Toren A, Kiss A, Fowler RA. Eligibility criteria of randomized controlled trials published in high-impact general medical journals: a systematic sampling review. JAMA 2007; 297 (11) 1233-1240
    CrossrefPubMedSearch in Google Scholar
  • 36 Zhang H, He Z, He X. et al. Computable eligibility criteria through ontology-driven data access: a case study of hepatitis C virus trials. AMIA Annu Symp Proc 2018; 2018: 1601-1610
    PubMedSearch in Google Scholar
  • 37 Averitt AJ, Weng C, Ryan P, Perotte A. Translating evidence into practice: eligibility criteria fail to eliminate clinically significant differences between real-world and study populations. NPJ Digit Med 2020; 3: 67
    CrossrefPubMedSearch in Google Scholar
  • 38 Ash N, Ogunyemi O, Zeng Q, Ohno-Machado L. Finding appropriate clinical trials: evaluating encoded eligibility criteria with incomplete data. Proc AMIA Symp 2001; 27-31
    PubMedSearch in Google Scholar
  • 39 Gehring S, Eulenfeld R. German medical informatics initiative: unlocking data for research and health care. Methods Inf Med 2018; 57 (S 01): e46-e49
    Thieme ConnectPubMedSearch in Google Scholar
  • 40 Baillie Gerritsen V, Palagi PM, Durinx C. Bioinformatics on a national scale: an example from Switzerland. Brief Bioinform 2019; 20 (Suppl. 02) 361-369
    CrossrefPubMedSearch in Google Scholar
  • 41 Prokosch H-U, Acker T, Bernarding J. et al. MIRACUM: medical informatics in research and care in university medicine. Methods Inf Med 2018; 57 (S 01): e82-e91
    Thieme ConnectPubMedSearch in Google Scholar