Uptake of an Integrated Electronic Questionnaire System in Community Pediatric Clinics

Abstract

Objective The study aimed to evaluate an integrated electronic questionnaire system implementation in outpatient community pediatric practices on workflow, completion rates, and recorded scores.

Methods We evaluated the implementation and outcomes of an integrated electronic questionnaire system at 45 community pediatric practices that used standardized questionnaires to screen for autism, depression, and substance use and to measure asthma control. Electronic health record (EHR) data for all well child visits were extracted for the 3 months before and after implementation. We used statistical process control charts to evaluate questionnaire completion rates and Chi-square tests to evaluate screening completion and positive screening rates. The collection and entry of questionnaire information was observed and timed.

Results EHR data included 107,120 encounters across 45 practices that showed significant and sustained improvement in completion rates for all questionnaires. The rate of recorded concerning questionnaires decreased for asthma control (19.3 vs. 12.8%, p < 0.001), stayed the same for autism (96.6 vs. 96.2%, p = 0.38), decreased for depression (9.5 vs. 6.7%, p ≤ 0.001), and increased for any substance use (9.8 vs. 12.8%, p < 0.001). Twelve practices were observed, and patient time and staff time managing questionnaires were decreased after implementation.

Discussion Electronic questionnaire administration saved staff time and patient time. We report overall improvement in questionnaire completion rates, with notable variation in improvement in completion across practices and in change in concerning recorded result rates across measures.

Conclusion Conversion of four standard paper questionnaires to an integrated electronic system reduces patient and staff time while increasing completion rates when well integrated into routine care.

Keywords

asthma - autism spectrum disorder - depressive disorder - surveys and questionnaires - electronic health records

Protection of Human and Animal Subjects

This project was approved by the UPMC Quality Improvement Review Committee (1629); projects approved by this committee do not meet the definition of human subjects research and therefore do not require institutional review board approval.

Publication History

Received: 17 November 2020

Accepted: 26 February 2021

Article published online:
14 April 2021

• References

• 1 American Academy of Pediatrics. Bright Futures/AAP recommendations for preventive pediatric health care (Periodicity Schedule) Updated: March, 2019. Accessed January 2020 at: https://www.aap.org/en-us/documents/periodicity_schedule.pdf
• 2 Centers for Medicare & Medicaid Services. Core set of children's health care quality measures for Medicaid and CHIP (Child Core Set). 2019 ; Accessed January 2020 at: https://www.medicaid.gov/medicaid/quality-of-care/downloads/medicaid-and-chip-child-core-set-manual.pdf
• 3 Dinakar C, Chipps BE. Section on Allergy and Immunology, Section on Pediatric Pulmonology and Sleep Medicine. Clinical tools to assess asthma control in children. Pediatrics 2017; 139 (01) e20163438
• 4 Van Cleave J, Kuhlthau KA, Bloom S. et al. Interventions to improve screening and follow-up in primary care: a systematic review of the evidence. Acad Pediatr 2012; 12 (04) 269-282
• 5 Hirai AH, Kogan MD, Kandasamy V, Reuland C, Bethell C. Prevalence and variation of developmental screening and surveillance in early childhood. JAMA Pediatr 2018; 172 (09) 857-866
• 6 U.S. Department of Health and Human Services, Health Resources and Services Administration, Maternal and Child Health Bureau. Child health USA 2014. Rockville, Maryland: U.S. Department of Health and Human Services; 2015 ; Accessed January 2020 at: http://mchb.hrsa.gov/chusa14
• 7 Snyder CF, Aaronson NK, Choucair AK. et al. Implementing patient-reported outcomes assessment in clinical practice: a review of the options and considerations. Qual Life Res 2012; 21 (08) 1305-1314
• 8 Foster A, Croot L, Brazier J, Harris J, O'Cathain A. The facilitators and barriers to implementing patient reported outcome measures in organisations delivering health related services: a systematic review of reviews. J Patient Rep Outcomes 2018; 2 (01) 46
• 9 Haverman L, Engelen V, van Rossum MA, Heymans HS, Grootenhuis MA. Monitoring health-related quality of life in paediatric practice: development of an innovative web-based application. BMC Pediatr 2011; 11 (01) 3
• 10 Schepers SA, Sint Nicolaas SM, Haverman L. et al. Real-world implementation of electronic patient-reported outcomes in outpatient pediatric cancer care. Psychooncology 2017; 26 (07) 951-959
• 11 Schepers SA, Sint Nicolaas SM, Maurice-Stam H. et al. First experience with electronic feedback of the Psychosocial Assessment Tool in pediatric cancer care. Support Care Cancer 2017; 25 (10) 3113-3121
• 12 Schepers SA, Haverman L, Zadeh S, Grootenhuis MA, Wiener L. Healthcare professionals' preferences and perceived barriers for routine assessment of patient-reported outcomes in pediatric oncology practice: moving toward international processes of change. Pediatr Blood Cancer 2016; 63 (12) 2181-2188
• 13 Barthel D, Fischer KI, Nolte S. et al. Implementation of the Kids-CAT in clinical settings: a newly developed computer-adaptive test to facilitate the assessment of patient-reported outcomes of children and adolescents in clinical practice in Germany. Qual Life Res 2016; 25 (03) 585-594
• 14 Robins DL, Casagrande K, Barton M, Chen CM, Dumont-Mathieu T, Fein D. Validation of the modified checklist for Autism in toddlers, revised with follow-up (M-CHAT-R/F). Pediatrics 2014; 133 (01) 37-45
• 15 Levy S, Weiss R, Sherritt L. et al. An electronic screen for triaging adolescent substance use by risk levels. JAMA Pediatr 2014; 168 (09) 822-828
• 16 Schatz M, Sorkness CA, Li JT. et al. Asthma Control Test: reliability, validity, and responsiveness in patients not previously followed by asthma specialists. J Allergy Clin Immunol 2006; 117 (03) 549-556
• 17 Nathan RA, Sorkness CA, Kosinski M. et al. Development of the asthma control test: a survey for assessing asthma control. J Allergy Clin Immunol 2004; 113 (01) 59-65
• 18 Jia CE, Zhang HP, Lv Y. et al. The Asthma Control Test and Asthma Control Questionnaire for assessing asthma control: Systematic review and meta-analysis. J Allergy Clin Immunol 2013; 131 (03) 695-703
• 19 Utter GH, Maier RV, Rivara FP, Nathens AB. Outcomes after ruptured abdominal aortic aneurysms: the “halo effect” of trauma center designation. J Am Coll Surg 2006; 203 (04) 498-505
• 20 Marrie TJ. The halo effect of adherence to guidelines extends to patients with severe community-acquired pneumonia requiring admission to an intensive care unit. Clin Infect Dis 2005; 41 (12) 1717-1719
• 21 Gnambs T, Kaspar K. Disclosure of sensitive behaviors across self-administered survey modes: a meta-analysis. Behav Res Methods 2015; 47 (04) 1237-1259
• 22 Bradford S, Rickwood D. Psychosocial assessments for young people: a systematic review examining acceptability, disclosure and engagement, and predictive utility. Adolesc Health Med Ther 2012; 3: 111-125
• 23 Iturralde E, Adams RN, Barley RC. et al. Implementation of depression screening and global health assessment in pediatric subspecialty clinics. J Adolesc Health 2017; 61 (05) 591-598
• 24 Anderson LM, Papadakis JL, Vesco AT. et al. Patient-reported and parent proxy-reported outcomes in pediatric medical specialty clinical settings: a systematic review of implementation. J Pediatr Psychol 2020; 45 (03) 247-265
• 25 McCambridge J, Witton J, Elbourne DR. Systematic review of the Hawthorne effect: new concepts are needed to study research participation effects. J Clin Epidemiol 2014; 67 (03) 267-277
• 26 Jensen RE, Chan KS, Weiner JP, Fowles JB, Neale SM. Implementing electronic health record-based quality measures for developmental screening. Pediatrics 2009; 124 (04) e648-e654