CC BY-NC-ND 4.0 · Gesundheitswesen 2024; 86(S 04): S275-S281
DOI: 10.1055/a-2325-0194
Original Article

Implementation of a telemedical urgency assessment procedure in the pediatric emergency room: evaluation results

Article in several languages: English | deutsch
Angelika Beyer
1   Institut für Community Medicine, Universitätsmedizin Greifswald, Greifswald, Germany
,
Kilson Moon
1   Institut für Community Medicine, Universitätsmedizin Greifswald, Greifswald, Germany
,
Thomas Hirsch
2   Klinik für Kinder- und Jugendmedizin, Universitätsmedizin Greifswald, Greifswald, Germany
,
Holger N. Lode
3   Klinik und Poliklinik für Kinder- und Jugendmedizin, Universitätsmedizin Greifswald, Greifswald, Germany
,
Wolfgang Hoffmann
1   Institut für Community Medicine, Universitätsmedizin Greifswald, Greifswald, Germany
,
Neeltje van den Berg
1   Institut für Community Medicine, Universitätsmedizin Greifswald, Greifswald, Germany
› Author Affiliations
 

Abstract

Background In rural areas in Germany, the number of emergency departments with pediatric expertise decreases. Telemedicine solutions are used sporadically, but they lack certain parameters for assessing a child's health status, such as touch and smell. We tested and evaluated the implementation of a telemedical, cross-hospital urgency assessment in pediatric emergency rooms. The telemedical urgency assessments were carried out via video conferences and were compared to the usual on-site procedure. Primary results of the concordance analysis have been published elsewhere. This work describes the results of the evaluation of the implementation.

Methods The telemedical urgency assessment was carried out in 5 pediatric emergency departments during the years 2015-2019. Various methods were used to evaluate the implementation. The following reports are based on (a) a parent questionnaire with two statements to be evaluated (entire project duration), (b) a survey of the physicians using telemedicine after each case (entire project duration) and (c) detailed process documentation (July 2017 until end of the project).

Results A total of 266 patients under 18 years old, recruited from four hospitals, were included in the study. (a) 210 parents completed the questionnaire. 78% of the parents felt adequately cared for and 70% could imagine telemedicine becoming established as a future supplementary care procedure. (b) The physicians' questionnaires for the telemedicine site were completed in 232 cases (87%). The average satisfaction rating was 1.8 on a 6-point-likert-scale (95% confidence interval: 1.64; 1.95). (c) The most frequent implementation problem concerned the technical implementation of the video conference. The evaluation of the accompanying documentation revealed in particular implementation barriers in the technical area (e. g. limited video and/or audio quality) and in the provision of human resources.

Conclusion Despite implementation barriers, the project showed that telemedical urgency assessment in acute pediatric care is a promising option for supporting care. Most of the participating clinicians needed a high level of support, which in some cases indicated a rather low level of digital competence. Increasing acceptance of telemedicine functionalities requires changes in society as a whole with improved framework conditions.


#

Introduction

Pediatrics and telemedicine

Worldwide and also in Germany, the use of telemedicine solutions in pediatrics has increased significantly in recent years. The international literature contains a large number of reviews on the topic of telemedicine and pediatrics with a specific disease focus such as headache [1] [2] [3], palliative care [4], treatment options for diabetes mellitus [5] [6] as well as in nephrology/urology [7] [8] und dermatology [9] [10] [11].


#

Acute pediatrics and telemedicine

There are also possibilities and concepts for the use of telemedicine in pediatric emergency and acute care. They are needed to improve or maintain equity in healthcare, especially in rural areas [12]. That this is possible is confirmed by Mitra et al., who found a high significant impact of telemedicine concepts on access to specialized care, on cost savings and on patient satisfaction [13]. However, the implementation and perpetuation of telemedicine concepts in pediatrics is associated with particular challenges worldwide, as senses such as smell and touch have to be adequately replaced for in some cases [12]. This problem is also described by Yao et al. in their scoping review of telemedicine-based physical examinations, which included 74 studies, of which only 5% concerned pediatrics [14]. Gutierrez et al. state that there is a high variability for telemedicine services that can effectively support smaller or rural hospitals in inpatient pediatric care. They conclude that their significant benefits are appreciated by rural and smaller hospitals, but further research is needed to evaluate clinical outcomes [15]. Dick et al. summarize in a review that there is little robust evidence for positive effects [16]. In the discussion of their rapid review, Fasolino et al. note broad acceptance, but this is not always accompanied by valuable mechanisms for care [17]. A positive evaluation of the telemedical urgency assessment would be a first step towards an important addition to pediatric care in rural regions. Emergency departments in hospitals without a children's ward could assess acute cases (here: children and parents who visit the emergency department independently) with telemedical support and make a qualified decision on further treatment. This could make a relevant contribution to providing medical care for children and young people close to home.

The project design, sample and primary outcome (how often is there agreement in the urgency assessment of patients who are triaged both on site and by telemedicine?) have already been published elsewhere [18]. The concordance analysis showed substantial agreement with a Cohen's Kappa of 0.64. Consequently, it was concluded that telemedical urgency assessment can support acute pediatric care. The project was initiated against the background that in Germany, pediatric departments have been closing in smaller hospitals for several years, particularly in regions with a low population density, and the overall number of hospital beds is decreasing [19] [20] [21]. The reasons for this are often the lack of doctors and nurses in structurally weak regions and the hospitals' financial deficits [22].

The implementation was evaluated based on the following research questions:

Do parents/guardians feel adequately cared for in the emergency department during a telemedicine urgency assessment?

What attitudes do parents/guardians have towards the future establishment of video conferencing as a supplementary care measure?

How do participating physicians on the telemedicine side assess the special features of this type of urgency assessment in a pediatric emergency department?

What implementation barriers, particularly in the areas of “personnel” and “technology”, can be identified?


#
#

Methods

Patient recruitment

For inclusion in the study, patients under the age of 18 who visited one of the participating pediatric emergency departments were included. The conditions had to be met that both the participating physicians and the supporting specialists were present on site and that it was at least theoretically possible to reach participating physicians in another participating hospital for the telemedical urgency assessment. If these conditions were theoretically met, the parents/guardians were informed about the study and then asked to sign the informed consent form. Children and adolescents whose state of health, in the opinion of the pediatrician on duty on site, required immediate treatment were not included in the study. Parents/guardians who, in the opinion of the recruiting person on site (pediatrician, doctor or nurse), had too little knowledge of German language to give informed consent were not included.


#

Implementation of the urgency assessments

The project started at a university medical center and basic care hospital 37 km away. Over the course of the project, three other rural hospitals joint the project. The project organization had to be adapted several times, as two paediatric departments were closed during the project period. Over the entire project period (01.05.2015 to 31.09.2019), four primary and standard care hospitals in rural areas were involved in the recruitment of 266 patients. Doctors from the basic care hospitals took part in both the on-site urgency assessment and the telemedical assessment. Doctors from university pediatrics were only involved on the telemedical side. A total of 19 doctors were involved in the study, the majority of whom were active both on site and telemedically. Ten specialists in pediatrics were active on-site, as well as four doctors who were about to complete their pediatric specialist training. On the telemedicine side, 18 doctors were active, 14 of whom were specialists in pediatrics and four of whom were about to complete their pediatric specialist training. For more information see [18]. The doctors on site were always responsible for diagnosis and treatment. The two independent urgency assessments [first on site, then via video conferencing system (VCS)] were carried out using a modified, software-based version of the Paediatric Canadian Triage and Acuity Scale (PaedCTAS), which was used to document twenty parameters [23]. Some of these parameters were reported during the telemedical consultation by a healthcare professional providing on-site support; the on-site healthcare professionals were nurses or doctors who had just started their practical year. The on-site supporting healthcare professionals did not carry out an urgency assessment, but provided support with the organization on site and during the telemedical video conference. The specific procedure is described elsewhere [18]. In case of recruitment or telemedical urgency assessment the non-university hospitals were paid a lump-sum expense allowance to finance the additional time required on the basis of a cooperation agreement.


#

Evaluation of the telemedical video conferences

(a) The parents/guardians were asked for a short, standardized assessment after the telemedical urgency assessment. The paper questionnaire was completed anonymously and contained two statements that had to be rated on a 6-point Likert scale (1=“strongly agree” to 6=“strongly disagree”). The two statements were “I felt adequately cared for during the telemedical consultation (video conference)” and “I can imagine telemedical consultations (video conferences) being established as an additional care procedure for rural regions”. In order to minimize socially desirable response behaviour, the parent questionnaires were handed over after the examination had been completed. Parents with little knowledge of German language were asked whether they understood the content; if not, they did not take part in the survey. The questionnaire was placed in a sealed letterbox marked for this purpose by the respondents themselves.

(b) The evaluation of the telemedical urgency assessment from the perspective of the physicians was carried out using a standardized questionnaire, which was part of the web-based documentation software as an electronic Case Report Form (eCRF). [Fig. 1] shows the entry mask for the eCRF. The first question was about general satisfaction with the course of the telemedical video conference. This was based on a 6-point Likert scale (1=“strongly agree” to 6=“strongly disagree”), followed by a text field for comments. In the further course of the evaluation, problems that arose during the video conferences were initially recorded in a standardized manner (see [Fig. 1]). In order to record the problems in more detail, filter questions were programmed, i. e. if “yes” was clicked for one of the problems mentioned, further fields were opened to specify the problem. The last question related to whether the telemedical situation influenced the medical decisions.

Zoom Image

Both parent and physician questionnaires were analyzed using Stata version 17.0 SE (Copyright 1985-2021 StataCorp LLC, StataCorp 4905 Lakeway Drive College Station, Texas 77845 USA). Depending on the response format, absolute and relative frequencies or average values with 95% confidence intervals were reported. Free texts were qualitatively analyzed with MAXQDA 24 and are reported in a linguistically and orthographically smoothed form.

(c) A further component of the evaluation was detailed accompanying process documentation with continuous data monitoring by the project coordinators. This documentation was kept from July 2017 until the end of the study and contained emails and short documentation entries of personal and telephone contacts with the project partners and involved professionals in the participating hospitals, including the IT administrators. The aim of this documentation was to describe problems during implementation in a comprehensible manner at a later date. Problems were operationalized as barriers that hindered or complicated the implementation of the telemedical urgency assessment.


#
#

Results

(a) Results from the paper questionnaires for parents/guardians

[Table 1] shows the evaluation of the telemedical situation by the accompanying parents/guardians. The first statement “I felt adequately cared for during the telemedical consultation (video conference)” was rated on a Likert scale of 1-6 with an average of 1.9 (n=211; 95% confidence interval: 1.74; 2.10). The second statement “I can imagine telemedical consultations (video conferences) being established as an additional care procedure for rural regions” was rated on a Likert scale of 1-6 with an average of 2.3 (n=212; 95% confidence interval: 2.06; 2.49).

Table 1 Parents' ratings of the telemedical urgency assessment, number (percent), n=210

I felt adequately cared for during the telemedical consultation (video conference).

Rating*

1

2

3

4

5

6

Number (%)

115 (54.5)

50 (23.7)

15 (7.1)

18 (8.5)

5 (2.4)

8 (3.8)

I can imagine telemedical consultations (video conferences) being established as an additional care procedure for rural regions.

Rating*

1

2

3

4

5

6

Number (%)

94 (44.3)

56 (26.4)

16 (7.6)

19 (9.0)

10 (4.7)

17 (8.0)

*6-point Likert scale: 1=„strongly agree“ bis 6=„strongly disagree“


#

(b) Results from the eCRF questionnaires for telemedicine physicians

The telemedicine physicians completed the evaluation eCRFs in 232 cases (87.2%) from a total of 266 recruited patients.

Course of the video consultation: Of the 232 evaluable cases, 55.2% (n=128) of the respondents selected the answer option “strongly agree” and 27.6% (n=64) “agree” for the statement “Overall, I was satisfied with the course of the telemedical consultation”. In 13.8% (n=29), the two middle ratings were given (“somewhat agree” or “somewhat disagree”) and for 3.4% of cases (n=8) the statement “disagree” or “strongly disagree” applied. This means that the question on satisfaction was rated an average of 1.8 on a Likert scale of 1-6 (95% confidence interval: 1.64; 1.95). In the text field following this statement, comments on the process were possible. This was used in 26.6% of the cases (n=69), in some cases several topics were noted. In 15 cases, the problems related to a lack of examination options (e. g. “auscultation not possible”, “no ear inspection possible”, “neurological examination useful”), in four cases to difficult communication options (“different opinion to colleague on site”, “difficult communication”), in 32 cases to problems with the video system (“camera does not focus”, “stitches cannot be shown sufficiently well in the video”, “skin cannot be assessed well”) and in five cases with the audio system (“sound quality limited – child's speech not understandable”, “insufficient acoustics during transmission”) of the conference system and in six cases to problems with the documentation system (“the child could not be entered in the system”, “difficult to call patients”). A complete list of comments can be found in the online supplement.

Technical Problems:  In 36 cases, respondents indicated that technical problems occurred during the consultation. Of these, “connection could not be established” was ticked four times, “disconnection during the consultation” twice, “insufficient audio quality” eleven times and “insufficient video quality” in 24 cases (multiple answers were possible). There was a problem with the operation of the video conferencing system in 14 cases. The “other problem” button was selected nine times. In the corresponding free texts (multiple answers were possible), two refer to insufficient examination options (“Inspection of the nasal passage not sufficiently possible by telemedicine”; “Patient must sit very still, camera must always be readjusted”), three refer to problems in handling the documentation system (“Could not start the system, not familiar with it”; “Software bulky – not very intuitive – moderate user-friendliness”; “Data record could not be processed during the examination period 10.7. due to technical problems on the ward”). Four further times it was mentioned that it was not possible to enter data into the documentation system due to unspecified problems – in these cases, the data set was created retrospectively by project staff. In one case, there was a problem with the remote control (“remote control had “hung up” – later documented cause was a empty battery).

Non-technical problems: In 13 cases, it was stated that problems of a non-technical nature occurred during the consultation. Of these, “Insufficient cooperation of the child not” was ticked three times. The corresponding comments were “crying child, proper measurement of vital signs not possible” and “child quite uncooperative, questionable localization of pain (lower right abdominal) so not better to determine” as well as “constantly scurrying around mother cannot calm child”. The fact that the mother was unable to calm her child was also selected once for the case that “Insufficient cooperation of the parents/caregivers”. The “other problem” button was selected four times for this. In the corresponding free texts, it was noted that “Everything is currently mixed up and under discussion as to when to enter what, the consultation suffered as a result”, “Ear pain was indicated by patient on pain scale with 8 (scale 0-10) while beaming, due to pain classification in triage at level I, which of course is not clinically correct”, “Reflex status could not be checked”, “Software”.

Influence on the medical decision: The question of whether the telemedical situation had an influence on medical decisions was answered with yes for 51 cases (22.0% of n=232 cases assessed). The free texts written in response to this question referred in two cases to the difficulty of communication (“Direct contact with the family useful, parents seemed less willing to talk”, “direct communication necessary”) and in only four cases to problems with the telemedical technology (e. g. “skin coloration difficult to assess”). The most frequently mentioned problem here was the lack of examination methods (n=33) to reliably assess the situation, often in relation to the desire for a more correct diagnosis or treatment recommendation (e. g. “only suspected diagnosis possible”, “examination of the abdomen would have been important” or “clinical examination not satisfactory”). A complete list of comments can be found in the online supplement.


#

(c) Results from the accompanying process documentation

The uncut raw version of the accompanying documentation comprises 107 pages (57,480 words) with entries on 59 different days. Barriers from the implementation phase are summarized below, particularly in the areas of personnel and technology.

Topic „personell“

One barrier that was discussed several times with several doctors and nurses were legal concerns about telemedicine. Not all doctors and nurses were willing to participate in the project, which made coordination difficult. Arguments against participating in the study ranged from clearly expressed professional policy concerns (e. g. “We're shooting ourselves in the foot with this”) to a lack of time both for recruiting patients on site and for carrying out the telemedical urgency assessment (e. g. “I can't do this as well”), to a different set of priorities (e. g. “I don't have time for this”).

In some departments, cooperation between doctors and nursing staff was generally problematic, as there was a lack of willingness to coordinate organizationally. However, this was necessary as the doctors on-site had to assess the situation first and then the supporting healthcare professionals had to act on site to organize and carry out the telemedical consultation.

The training courses on how to use both the documentation software and the VCS were almost always individual training sessions. VCS were not widely used in healthcare before the pandemic, which meant that participants almost always had to learn how to use them from scratch. In addition, many were unfamiliar with the use of the virtual private network (VPN) tunnel for the case documentation as well. Here, tech-savvy participants were generally more positive and learned quickly, but the majority of participants needed several refresher courses.

In two hospitals, workflows had to be changed as there was no nursing staff available in the emergency departments for the project. Supporting healthcare professionals had to be found to inform the parents and recruit the children and adolescents and then organize and support the video conference. This was achieved with doctors who had just started specialist training in the participating clinic. However, this arrangement led to a significant restriction of the possible recruitment times of a maximum of twice four hours/week at best. By the end of the project, it had not been possible to establish the regular availability of a pediatrician for telemedical consultations. This would also have required the reorganization of work processes, for which insufficient resources were available.


#

Topic “technology”

Case documentation was carried out using password-protected Java-based documentation software that could be accessed via login. In general, networking was a major difficulty for the hospitals, some of which were not structurally connected, as IT departments have often been outsourced or centralized within corporate groups in recent years. In some cases, firewalls had to be reconfigured. The coordination required to set up the VPN tunnels required an unexpectedly high degree of frustration tolerance, communication and cooperation between pediatric chief physicians, hospital IT staff and project staff.

For the video conferences, some of the hardware and software had to be reinstalled and reconfigured to ensure compatibility with the systems in the other hospitals. Separate cables/sockets were required to ensure connection stability. This meant that an additional room had to be made available for the technical equipment. This was also a major barrier to implementation, as in some hospitals this room was located in an emergency room away from the ward, where space was already limited. This in turn meant that telemedicine staff had to make extra trips outside the ward. In addition to the case documentation system, the VCS could only be used bilaterally. In one of the hospitals, it could only be started from there with additional organizational effort.


#
#
#

Discussion

Despite considerable barriers to implementation, it was shown that telemedical urgency assessment in acute pediatric care is a promising option for supporting and supplementing care. The acceptance of parents/guardians was clearly positive. The standardized evaluation survey via eCRF also resulted in 83% of physicians using telemedicine being either completely or largely satisfied. The accompanying process documentation pointed to a number of barriers that could not always be overcome. This can also be found in other studies and is one of the conclusions of the review by Saidinejad [12], To our knowledge, no studies have yet been conducted and published in Germany. Some of the barriers in this project can be attributed to the fact that a significant number of doctors and nurses were not very willing to participate in the project. Secondly, the participants required multiple technical training sessions and were not very tech-savvy. Videoconferencing-related technical skills and attitudes towards telemedicine are also developing very cautiously in other projects [24]. Due to this selected group of doctors, the rather positive assessment of telemedicine in the project described here is probably due to a selection bias.

However, some barriers could also be overcome. The decisive supporting factor here was the fundamentally positive attitude of the chief physicians involved in giving telemedicine a chance. In the time before the Covid pandemic, there were a considerable number of pediatricians who could hardly imagine telemedicine in acute pediatric care because, e. g. their own sense of smell or touch could not be used. The pandemic, with its high need for isolation and the associated restrictions on the movement of patients and families, has led to a rethink among many doctors. However, there is still a need to expand and consolidate the greater willingness to use telemedicine with good training and further education concepts. This applies in particular to (tele)communication skills and the question of how physical examinations can be meaningfully supported by an exchange with the specialist on site, despite the lack of possibilities to use one's own sensory organs. This is confirmed by the recently published reviews by Yao and Dick [14] [16].

Overcoming organizational barriers, which are confirmed in the reviews by Fasolino et al. [17] and Pettit, Young and Jung [7], requires not only a positive attitude on the part of those involved, but also changes in society as a whole that affect the healthcare system and digitalization.

The study has some limitations. Since this project is a study in a real care setting, the design and processes had to be adapted to the organizational processes in the hospital. The project was carried out in hospitals and with doctors who were explicitly interested in telemedicine solutions. This means that no representative data is available for a specific group of patients or for a specific region. However, it can be assumed that the participating emergency departments in rural hospitals are typical emergency departments with typical cases.

The incomplete number of evaluation eCRFs could have led to an overestimation of positive evaluation results. The incomplete documentation on the one hand is due to the fact that the accompanying process documentation was only implemented from July 2017. Secondly, it follows that technical and/or personnel problems and/or the availability of the room in which the telemedicine equipment was installed could have led to the termination of video conferences and thus also the incomplete documentation. Documentation was then only provided by the project staff in individual cases if the doctors reported this to them afterwards. Only complete data sets were included in the evaluation presented here.

In addition, no data was collected from people who did not take part in the study. This would theoretically have been essential for the description of an implementation, but cannot be done in retrospect.

A further limitation is that no analyses could be carried out due to the anonymous survey of parents' assessments and the resulting inability to assign the assessments to individual cases as well as the frequent lack of socio-demographic information. The conclusion must therefore remain general at this point. In a future project, it would be desirable to collect data in such a way that such analyses can be carried out, e. g. whether the attitude of parents/guardians and doctors towards telemedicine depends on the age of the patients treated or whether these assessments were concordant between the various participants on a case-by-case basis.

One of the strengths of the study is that the interviews with those involved were not conducted once, but for each individual case, so that barriers could be identified in different situations. Another strength is that the study was conducted in real care settings with real patients. The results can therefore be easily mapped to real-life care.


#

Conclusion

The evaluation results of the implementation process support the results of the concordance analysis [18]. Although not all problems could be solved by the end of the project, there was a high level of acceptance of and satisfaction with telemedicine when telemedical consultations were carried out to assess urgency. At this point, some time after the pandemic, a clear increase in acceptance of telemedicine can be observed, which goes hand in hand with a growing willingness to overcome existing obstacles. Technical developments must help to optimize the tools and processes and facilitate the integration of telemedicine concepts into the workflows of hospitals and emergency departments. In rural regions in particular, but not exclusively, there is a need for scientific support for the implementation of these instruments and processes. This can contribute to solving the expected care deficits.


#

Ethical considerations

The project has a positive vote from the Ethics Committee of the University Medicine Greifswald and is registered in the German Register of Clinical Trials (ID: 00013207).

This article is part of the DNVF supplement “Health Care Research and Implementation”


#
#

Acknowledgement

Special thanks go to the medical doctors and nurses who were involved and to the parents and their children who took the extra time and patience for the study. Thanks also to the reviewer for the intensive feedback which helped us to improve the manuscript significantly.

Supplementary Material

Zusätzliches Material


Correspondence

Angelika Beyer
Universitätsmedizin Greifswald Institut für Community MedicineEllernholzstr. 1-2
17487 Greifswald
Germany

Publication History

Accepted Manuscript online:
13 May 2024

Article published online:
15 August 2024

© 2024. The Author(s). This is an open access article published by Thieme under the terms of the Creative Commons Attribution-NonDerivative-NonCommercial-License, permitting copying and reproduction so long as the original work is given appropriate credit. Contents may not be used for commercial purposes, or adapted, remixed, transformed or built upon. (https://creativecommons.org/licenses/by-nc-nd/4.0/).

Georg Thieme Verlag
Rüdigerstraße 14, 70469 Stuttgart, Germany


Zoom Image
Zoom Image
Abb. 1 Eingabemaske der Dokumentationssoftware zur Evaluation der Videokonferenz (bei Anklicken von Ja öffneten sich detaillierte Filterfragen).