Comparison of Acceptance and Knowledge Transfer in Patient Information Before an MRI Exam Administered by Humanoid Robot Versus a Tablet Computer: A Randomized Controlled Study

Abstract

Purpose To investigate whether a humanoid robot in a clinical radiological setting is accepted as a source of information in conversations before MRI examinations of patients. In addition, the usability and the information transfer were compared with a tablet.

Methods Patients were randomly assigned to a robot or tablet group with their consent prior to MRI. The usability of both devices was compared with the extended System Usability Scale (SUS) and the information transfer with a knowledge query. Reasons for refusal were collected by a non-responder questionnaire.

Results At the University Hospital Halle 117 patients were included for participation. There was no statistically significant difference in gender and age. Of 18 non-responders, 4 refused to participate partly because of the robot; for another 3 the reason could not be clarified. The usability according to SUS score was different with statistical significance between the groups in the mean comparison and was one step higher for the tablet on the adjective scale. There was no statistically significant difference in knowledge transfer. On average, 8.41 of 9 questions were answered correctly.

Conclusion This study is the first application, in a clinical radiological setting, of a humanoid robot interacting with patients. Tablet and robot are suitable for information transfer in the context of MRI. In comparison to studies in which the willingness to interact with a robot in the health care sector was investigated, the willingness is significantly higher in the present study. This could be explained by the fact that it was a concrete use case that was understandable to the participants and not a hypothetical scenario. Thus, potentially high acceptance for further specific areas of application of robots in radiology can be assumed. The higher level of usability perceived in the tablet group can be explained by the fact that here the interface represents a form of operation that has been established for years in all population groups. More frequent exposure to robots could also improve the response in the future.

Key Points:

• patients accept humanoid robots in clinical radiologic situations

• at present they can only convey information as well as an inexpensive tablet

• future systems can relieve the burden on personnel.

Citation Format

• Stoevesandt D, Jahn P, Watzke S et al. Comparison of Acceptance and Knowledge Transfer in Patient Information Before an MRI Exam Administered by Humanoid Robot Versus a Tablet Computer: A Randomized Controlled Study. Fortschr Röntgenstr 2021; 193: 947 – 954

Key words

robotics - health communication - magnetic resonance imaging - information services - randomized controlled trial

Introduction

Robots are used broadly in the industrial sector to automate processes [1] [2] since the use parameters and interaction with people, which is usually minimal, are clearly defined [3] [4] [5] and the development and implementation of systems entail minimal obstacles. Robots have increasingly become a topic of interest among the public in recent years. There is currently no fundamental and universal definition of the term “robot” [6] [7].

Outside industrial applications, humanoid robots are used for support, for example for the sale of electronics or in the entertainment industry [8] [9]. The use of robots in the health care system also seems reasonable. However, only 55 % of those surveyed in an international questionnaire including 11 000 participants were open to the use of “artificial intelligence” and robots for their own health care needs [10]. In a Eurobarometer survey conducted in 2012 in 27 European countries, 57 % of the 26,751 participants responded that they would be uncomfortable undergoing an operation performed by a robot [11] [12]. However, in smaller but more specific surveys regarding robots in the health care system, 63 % of respondents stated that they are not afraid and have no reservations regarding robots [13]. Use of a humanoid robot at hospitals reported to date is limited to the reception area for such tasks as acting as a greeter and providing information [14]. The use of robots for treatment purposes represents a special case. Therefore, robots were used, for example, as therapeutic tools in a study in pediatric oncology to reduce fear, anger, and depression [15], in various settings in children on the Autism spectrum [16], and for personalized patient instruction for children with diabetes [17]. There are also approaches for including corresponding systems in the training of chronically ill patients [17] [18]. In the context of radiology, we are not aware of any regular use of humanoid robots involving patient contact.

In radiology, the use of robots for therapeutic applications as well as information transfer as part of the informed consent discussion would be conceivable: Information transfer prior to diagnostic examinations like CT and MRI is a standardized work step that can already be performed digitally [19]. Although the informed consent discussion is a non-delegable task that must be performed by a physician (informed consent requirement § 630e of the German Civil Code), an information sheet can be sufficient in routine interventions as long as patients have the opportunity to discuss any unanswered questions with the treating physician. Prior to diagnostic MRI examinations, the corresponding information is often also provided in the form of an information sheet. Individual studies show that tablet-based provision of information is preferred to paper [20] or that the two methods are at least considered equal [19].

The use of humanoid robots would be a conceivable alternative to the use of tablet-based information transfer since the human-like aspects of the robot could improve patient attention and information reception. The advantages should be similar to those of table-based information recording and transfer compared to a paper-based method (higher completion rates and lower probability of information loss) [19]. Moreover, studies examining the fundamental acceptance of humanoid robots are needed to be able to better plan future areas of application of this technology (e. g., support for anxious patients during angiographic interventions).

Objective

Three questions were examined in the present study:

1. Evaluation of the acceptance of a humanoid robot as a provider of information in a clinical radiological setting,

2. User friendliness of the robot compared to a tablet with respect to patient information,

3. Comparison of the two methods of information transfer regarding the planned MRI examination.

Materials and Methods

Competent patients of legal age undergoing an elective diagnostic MRI examination were included in this prospective study. Patients with serious psychiatric diseases, significant pain, or suspicion of dementia were excluded. Further exclusion criteria included a German language proficiency level of less than C1, a monitoring requirement, and an MRI examination in the last month. The local ethics committee of medical faculty of the Martin-Luther-University Halle-Wittenberg approved the study (processing number 2018–2023).

The patients were selected during MRI registration according to the inclusion and exclusion criteria. The process and patient exclusion according to the CONSORT recommendations are shown in [Fig. 2]. Suitable patients in the waiting room were then asked if they would be interested in participating in the study. Those who expressed willingness to participate received written information about the study that was explained in greater detail upon request. After subjects gave their informed consent, they were randomly assigned to one of two groups – information provided by a humanoid robot or information provided by a tablet. Group assignment was performed on a daily basis since a technician had to be present to supervise the robot and a room with the necessary audio and camera equipment had to be available. Participants were blinded to their group allocation until the start of information transfer.

Fig. 2 Flowchart of included and excluded subjects. “Other reasons” included interpreter/other language barriers, urgent follow-up examinations, legal guardian, delirium, severe depression, delusional state, schizophrenia according to registration.

Depending on the group assignment, instead of receiving the usual information sheet, subjects received information from a humanoid robot or received the same information on a tablet with identical interactive content. After inclusion, there were no dropouts.

Patients who did not want to participate in the study were asked to complete a questionnaire regarding the reasons for their refusal to participate. 15 out of 18 people (83 %) took advantage of this opportunity, while 3 did not wish to make any statement. Multiple choice questions were used to record reasons. Responses could be grouped into two categories “reasons not related to the robot” and “reasons related to the robot”.

Sociodemographic data was collected for all study participants as well as patients who refused to participate in the study but completed the non-responder questionnaire.

After the intervention, the patients in both groups were asked to complete a multiple-choice questionnaire and an expanded system usability scale (SUS) [21] [22].

The SUS determines the subjective usability of a system as perceived by the user. The ten items include five positive and five negative statements and use a Likert scale of 0 to 4 points. The ten individual values then yield a maximum SUS score of 100 [21].

Systems can be considered usable if they reach a threshold value of 68 or higher [22]. In previous studies, an adjective scale was developed to make the SUS score easier to understand [23] ([Table 1]).

Table 1 Adjective rating scale for SUS Score cited by Bangor et. al. 2009.

SUS Score	acceptability range	adjective rating
90–100	acceptable	excellent till best imaginable
80–89		excellent
68–79		good
50–67	marginal	so-so
35–49	not acceptable	poor
0–34		worst imaginable

In addition to the ten SUS items, the questionnaire included five additional statements added specifically for the present study. These were used to evaluate how patients understood the presented information and whether they were comfortable with the interaction. Moreover, the questionnaire recorded whether patients preferred receiving information from robot, a film on an iPad, or a conversation with a radiologist.

Nine multiple-choice questions were used to assess the transfer of information in order to quantify a possible difference between the methods of information transfer. Therefore, for example, participants were asked which objects are allowed and which are prohibited near the MRI unit and what the general conditions for performing the examination are. The complete questionnaire is provided in the supplement.

The statistics and analysis program “IBM SPSS Statistics 25” was used to evaluate the questionnaires.

The robotic system “Pepper” was introduced by the Japanese telecommunications company “Softbank” in 2014 and is sold by “Softbank Robotics Aldebaran”. The primary functions of the 1.2-meter robot are communication and interaction via voice command and output as well as non-verbal communication via body language [24] [25]. To decrease negative reactions among users, the device has a humanoid, gender-neutral appearance based on the baby schema ([Fig. 1]) [26] [27]. A touchscreen located in the chest region was used during information transfer, for example, to show MRI images and an MRI unit.

Fig. 1 Humanoid robot providing patient with information.

The tablet used in the control group used a Node.js web application developed specifically for this study that provided the same content regarding patient information. The tablet and robot were used without an additional person being present in the room. A film showing the interaction with the robot is provided in [Video 1].

Results

In the period from October 2018 to February 2019, 135 patients were prospectively asked to participate in the study. 117 patients (87 %) agreed to participate. 65 participants were randomly included in the “robot” group and 52 in the “tablet” group ([Table 2]). 18 people (13 %) refused to participate (non-responder; NR).

Table 2 Description of participants and Non-Responder.

	participants (n = 117)		non-responder (n = 18)
	robot (n = 65)	tablet (n = 52)
gender
woman	28	20	10
men	36	32	8
missing value	1	0	0
age
18–20 years	3	1	0
21–30 years	4	6	0
31–40 years	11	6	2
41–50 years	9	7	1
51–60 years	13	22	2
61–70 years	15	4	8
71–80 years	6	6	4
81–90 years	1	0	1
91–100 years	0	0	0
missing value	0	0	0
education
not specified	1	2	1
no degree	2	0	2
secondary school	8	7	0
junior high school	36	20	4
high school	8	13	1
university	10	10	3
missing value	0	0	7

41 % of study participants were female and 58 % were male. 1 % did not specify. The average age was 51.3 years (SD 15.7). In the comparison between the robot and tablet groups, there was no statistically significant difference regarding sex (χ² (1, N = 117) = 1.138, p = 0.765) or age [df = 112] = 0.25, p = 0.804).

56 % of NRs were female and 44 % were male. Thus, the NRs did not differ significantly from the study participants (χ² (1, N = 134) = 1.28, p = 0.259). The average age of the NRs was 63.9 years (SD 12.6). In comparison, the average age of the study participants was 51.3 (SD 15.7) years. This age difference is statistically significant (t[df = 133] = 3.27, p = 0.001).

The highest level of education of 48 % of participants was high school (to grade 10) (robot group: 55 %; tablet group: 39 %) ([Table 2]). In the tablet group the next most common level of education was high school (25 %) followed by university (19 %) and secondary school (14 %). In the robot group, the second most frequent level of education was university (15 %) followed by high school and secondary school (each at 12 %). Three participants did not provide any information about their level of education and two stated that they had no diploma. One NR had no diploma and seven did not provide any information (44 %).

The refusal reasons of the 18 NRs could be allocated to four categories: “No reason given” (n = 3), “reasons not related to the robot” (n = 11), “reasons related to the robot” (n = 2), and both reasons in combination (n = 2). Thus four (3 %) of those surveyed refused to participate at least partly due to the robot. The reason could not be clarified in the case of another three respondents (2 %).

The reasons not related to the robot included “I feel too sick” (n = 6), “no time to participate in the study” (n = 2), and “I am already familiar enough with the MRI examination and do not need any further explanation” (n = 2).

With respect to the reasons related to the robot, two NRs stated that they “are afraid that robots could replace physicians and nurses” (n = 2). Each of the following reasons was specified by one NR: “I am afraid that the robot will not understand me, or I will not be able to understand it”, “I am afraid the robot will collect too much personal data”, “nonverbal conversation with the robot is not possible so that misunderstandings can arise” and “a conversation with a robot is too impersonal”.

A comparison of the mean SUS score between the tablet group (M = 89.26, SD 16.26) and the robot group (M = 75.00, SD 12.16) yielded a significant difference (t[df = 105] = –5.19; p < 0.001). Due to a violation of the normal distribution assumption (Kolmogorov-Smirnov Test, z = 0.152; p < 0.001), a nonparametric test of the difference between the groups was performed (Mann-Whitney Test p < 0.001).

As shown in [Table 3], an adjective rating scale was used to categorize the acceptability range. 10 of the 117 participants did not provide responses (robot group: n = 2; tablet group: n = 8).

Table 3 SUS Score on adjective rating scale.

acceptability range	adjective rating	robot (n = 65)		tablet (n = 52)		total (n = 117)
		n	%	n	%	n	%
acceptable	excellent till best imaginable	15	23,1	30	57,7	45	38,5
	excellent	12	18,5	6	11,5	18	15,4
	good	12	18,5	3	5,8	15	12,8
marginal[*]	so-so	21	32,3	5	9,6	26	22,2
not acceptable	poor	3	4,6	0	0,0	3	2,6
	worst imaginable	0	0,0	0	0,0	0	0,0
not specified	not specified	2	3,1	8	15,4	10	8,5

^* SUS Score lower than 68.

Based on the cut-off value recommended in the literature, an SUS score of 68 or higher denotes a device with sufficient and acceptable usability.

The robot was considered acceptable by 60 % of participants and the tablet by 75 %, with the majority of responses being concentrated in the category “excellent to best imaginable” (robot group: 23 %; tablet group: 58 %). In the case of an SUS score of less than 68, the values were concentrated in the “marginal” category (robot group: 32 %; tablet group: 10 %).

Regarding the statement in question 1 of the SUS as to whether patients would want to use the system regularly in the future for receiving information, 61 % (robot group: 55 %; tablet group: 67 %) were in agreement while 28 % (robot group: 34 %; tablet group: 21 %) were neutral.

87 % of participants (robot group: 85 %; tablet group: 90 %) felt safe interacting with the system (question 9). 7 % (robot group: 11 %; tablet group: 2 %) were neutral with respect to this statement. One person in each group disagreed with this statement.

92 % of participants (robot group: 94 %; tablet group: 90 %) indicated that they understood all information provided by the system (item 11).

76 % of participants (robot group: 77 %; tablet group: 75 %) indicated that they felt comfortable with the provision of information, while 17 % (robot group: 17 %; tablet group: 17 %) remained neutral.

37 % of participants in the robot group and 15 % in the tablet group agreed with the statement “I would prefer a discussion with a robot to watching a film on an iPad” (item 13). 23 % of participants in the robot group and 25 % in the tablet group did not agree with this statement. 37 % in the robot group and 54 % in the tablet group were neutral.

There was no significant difference in questions 1, 9, 11, 12, 13.

45 % of participants in the robot group and 25 % in the tablet group agreed with the statement “I would prefer a discussion (with the same content) with a radiologist” (item 14). 12 % of participants in the robot group and 21 % in the tablet group did not agree with this statement. 40 % in the robot group and 48 % in the tablet group were neutral. The described difference in the comparison of the mean values between the groups (T[df = 110] = 2.487; p = 0.014) was significant. Due to a violation of the normal distribution assumption (Kolmogorov-Smirnov Test, z = 1.655; p < 0.008), a nonparametric test of the difference was performed (Mann-Whitney Test U = 1158.5, Z = -2.398, p < 0.016).

A further goal of the study was to determine how effective each modality was at transferring knowledge. The majority of those surveyed (71 %) were able to correctly answer nine of nine questions (robot group: 72 %; tablet group: 69 %) ([Table 4]).

Table 4 Knowledge Objectification.

	robot (n = 65)	tablet (n = 52)	total (n = 117)
Count of correct answers (max. 9)
2	0	1	1
3	2	1	3
4	0	0	0
5	2	0	2
6	0	2	2
7	1	3	4
8	13	9	22
9	47	36	83
mean (SD)	8,46 (1,263)	8,35 (1,413)	8,41 (1,327)

On average, there were 8.46 (SD: 1.263) correct answers in the robot group and 8.35 (SD: 1.413) in the tablet group.

If the age of the participants is taken into consideration, those over 60 answered on average 8.16 (N = 114; SD: 1.526) questions correctly while the younger patients answered 8.50 (N = 114; SD: 1.260) questions correctly. The difference between the two age groups was not statistically significant (t(112) = 1.232, p = 0.221). If level of education is included, participants with a high school diploma or a higher degree answered an average of 8.51 (SD 1.325) questions correctly, while those with a 10-year high school diploma or a lower level of education answered an average of 8.36 (SD 1.334) questions correctly.

The difference between levels of education was also not statistically significant (t(117) = -0.609, p = 0.544).

Discussion

The present study is the first study and first known case in which a humanoid robot interacted with patients in the clinical radiological setting without being used as a therapeutic tool.

1. Acceptance

In the aforementioned PricewaterhouseCoopers (PwC) survey, 41 % of those surveyed in Germany stated that they would not be willing to discuss their health care needs with artificial intelligence or robots [10]. The Eurobarometer study shows similar results with 57 % of respondents stating that they would be uncomfortable with surgery performed by a robot. The willingness of those participating in this study was significantly higher: Only 3 % of patients refused to participate because of the robot. Even if patients who did not want to specify a reason are included, the possible refusal rate due to the robot is less than 5 %. The significantly higher acceptance in our study could be explained by the fact that it was a concrete use case while the participants in the aforementioned PwC survey were asked generally about situations in which they would accept robots (e. g., after a minor operation).

Due to the small group size, it was not possible to statistically examine the reasons for refusal as a function of age, sex, and level of education. The significantly higher age in the NR group could be due to a higher disease burden and a consequently reduced willingness to participate. Alternatively, a lower level of familiarity with technology should also be considered.

2. Usability

The greater ease of use perceived in the tablet group may be able to be explained by the fact that touchscreens have been widely used for years as a user interface by the entire population and tablets are therefore easier to use than previously unknown robotic systems with verbal interaction.

Reservations regarding regular use would presumably decrease as a result of longer and more frequent exposure to the corresponding technology in the context of information transfer and other aspects of health care [28] [29].

The fact that significantly more participants (45 %) in the robot group than in the tablet group (25 %) preferred human contact can also be explained by their minimal to nonexistent prior exposure. Another possible reason could be that the participants subconsciously felt that the robot was less reliable or creepy. However, this does not coincide with the data in the present study. In the direct survey, there was no difference regarding level of comfort between the two groups.

3. Knowledge transfer

A significant majority of participants subjectively indicated that they understood all information provided by the respective system with no significant difference between the groups.

In the objective evaluation of knowledge transfer, it was also able to be shown that the majority of patients understood the information correctly. Therefore, both systems are fundamentally suitable for transferring information that is relevant to MRI examination.

The unequal number of participants in the robot and tablet groups is due to the fact that the robot-based method was organized on a daily basis so that randomization could only be performed on a daily basis but not in a subject-based manner.

Limitations of the study:

With regard to the method, it must be stated that the questionnaire regarding the transfer of knowledge may have been too easy resulting in the relatively high number of correct answers in both groups.

A strength of the study is its randomized controlled design. All central study results were reported by the study participants so that an observer bias can be ruled out. Moreover, there were no dropouts, thereby minimizing the attrition bias.

Due to the small number of subjects, subgroup analyses (particularly among the subjects who refused to participate) could not be performed.

The purchasing cost of the robot system (approximately 10,000 to 20,000 Euros) and the need for the presence of a technician currently argue against regular use of the system, especially in light of the lack of improvement in information transfer.

Conclusion

1. In the present study, a large majority of the surveyed patients were willing to interact with a humanoid robot in a concrete health care situation. Willingness was higher than general surveys had indicated. This can be explained by the fact that the study had a concrete context that was understandable to participants (information about an upcoming diagnostic MRI examination in contrast to the use of robots in general). Therefore, potentially higher acceptance of further specific applications of robots in radiology can be assumed.

2. The usability of a tablet compared to a humanoid robot was considered slightly better which is probably due to the greater familiarity with tablets. However, greater exposure to humanoid robots could change this in the future. At present, robots are not a cost-effective alternative to tablets or information sheets.

3. A robot and a tablet are both suitable for providing information in the context of MRI.

Video 1 Film with sample dialog.

Conflict of Interest

The authors declare that they have no conflict of interest.

• Literatur

• 1 Bannat A, Gast J, Rehrl T. et al A Multimodal Human-Robot-Interaction Scenario: Working Together with an Industrial Robot. 5611. 2009: 303-311 DOI: 10.1007/978-3-642-02577-8_33
• 2 Cooper M, Keating D, Harwin W. et al Robots in the Classroom – Tools for Accessible Education. In: Assistive Technology on the Threshold of the New Millennium, Assistive Technology 1999: 448-452
• 3 Anderson RL, Gartner WB. When Robots and People Work Together. Robotics 1985; 1: 69-76 DOI: 10.1016/S0167-8493(85)90065-8.
• 4 Heyer C. Human-Robot Interaction and Future Industrial Robotics Applications. 2010: 4749-4754 DOI: 10.1109/IROS.2010.5651294
• 5 Moniz A, Krings BJ. Robots Working with Humans or Humans Working with Robots? Searching for Social Dimensions in New Human-Robot Interaction in Industry. Societies 2016; 6: 23 DOI: 10.3390/soc6030023.
• 6 Klein B, Graf B, Schlömer IF. et al Robotik in der Gesundheitswirtschaft. Einsatzfelder und Potenziale. Heidelberg: medhochzwei; 2018
• 7 Meißner A. Technisierung der professionellen Pflege. Einfluss. Wirkung. Veränderung. In: Hagemann T. Hrsg. Gestaltung des Sozial- und Gesundheitswesens im Zeitalter von Digitalisierung und technischer Assistenz. Forschung und Entwicklung in der Sozialwirtschaft. Nomos; 2017: 153-172 DOI: 10.5771/9783845279435-153
• 8 Boran M. Robot shop assistant Pepper makes US debut (18.12.2018). Im Internet (published 18.8.2016 access: 18.12.2018): https://www.irishtimes.com/business/technology/robot-shop-assistant-pepper-makes-us-debut-1.2757113
• 9 Kattan D. 4 real-life examples of robots in the workplace … that don’t want your job. Im Internet (Stand: 04.11.2018): https://immense.net/7048-24-real-life-examples-of-robots-in-the-workplace/
• 10 Arnold D, Wilson T. What doctor? Why AI and robotics will define New Health. Im Internet (Stand: 25.06.2018): https://www.pwc.com/gx/en/industries/healthcare/publications/ai-robotics-new-health/ai-robotics-new-health.pdf
• 11 Newsroom Editor. Eurobarometer 2012 Survey on Public Attitudes towards Robots [Report]. Im Internet (Stand: 20.12.2020): https://ec.europa.eu/commfrontoffice/publicopinion/index.cfm/Survey/getSurveyDetail/yearFrom/1974/yearTo/2012/surveyKy/1044
• 12 Loffredo D, Tavakkoli A. What are European Union Public Attitudes towards Robots?. Im Internet (Stand: 20.12.2020): https://pdfs.semanticscholar.org/b2ea/4ef7f21518f5f30fc0ca526bc3ff2fd6f855.pdf?_ga=2.41434722.1908020248.1579170775-143422030.1547635814
• 13 Broadbent E, Kuo IH, Lee YI. et al Attitudes and Reactions to a Healthcare Robot. Telemed J E Health 2010; 16: 608-613 DOI: 10.1089/tmj.2009.0171.
• 14 Thys J. Robot receptionists introduced at hospitals in Belgium (09.11.2018). Im Internet (published 14.6.2016 access: 9.11.2018.2018): https://www.theguardian.com/technology/2016/jun/14/robot-receptionists-hospitals-belgium-pepper-humanoid
• 15 Alemi M, Ghanbarzadeh A, Meghdari A. et al Clinical Application of a Humanoid Robot in Pediatric Cancer Interventions. Int J of Soc Robotics 2016; 8: 743-759 DOI: 10.1007/s12369-015-0294-y.
• 16 Coeckelbergh M, Pop C, Simut R. et al A Survey of Expectations About the Role of Robots in Robot-Assisted Therapy for Children with ASD: Ethical Acceptability, Trust, Sociability, Appearance, and Attachment. Sci Eng Ethics 2016; 22: 47-65 DOI: 10.1007/s11948-015-9649-x.
• 17 Blanson Henkemans OA, Bierman BPB, Janssen J. et al Using a robot to personalise health education for children with diabetes type 1: a pilot study. Patient Educ Couns 2013; 92: 174-181 DOI: 10.1016/j.pec.2013.04.012.
• 18 Al-Taee MA, Kapoor R, Garrett C. et al Acceptability of Robot Assistant in Management of Type 1 Diabetes in Children. Diabetes Technol Ther 2016; 18: 551-554 DOI: 10.1089/dia.2015.0428.
• 19 Raatschen H, Alikhani B, Grosser A. et al Erste Erfahrungen mit einer digitalen Patientenaufklärung im klinischen Alltag. Fortschr Röntgenstr 2017; 189: S1-S124 DOI: 10.1055/s-0037-1600183.
• 20 Schlechtweg PM, Hammon M, Giese D. et al iPad-based patient briefing for radiological examinations-a clinical trial. J Digit Imaging 2014; 27: 479-485 DOI: 10.1007/s10278-014-9688-x.
• 21 Brooke J. SUS. a ‘quick and dirty’ usability scale. In: Jordan PW, Thomas B, Weerdmeester BA. et al Hrsg. Usability Evaluation in Industry. Taylor & Francis; 1996: 189-194
• 22 Brooke J. SUS: A Retrospective. Journal of Usability Studies 2013; 8: 29-40
• 23 Bangor A, Kortum P, Miller J. Determining What Individual SUS Scores Mean. Adding an Adjective Rating Scale. Journal of Usability Studies 2009; 4: 114-123
• 24 Demir KA, Caymaz E, Elçi M. Issues in Integrating Robots into Organizations. 2017
• 25 Demir KA. Research Questions in Roboethics. Mugla Journal of Science and Technology 2017; 160-165 DOI: 10.22531/muglajsci.359648.
• 26 Borgi M, Cogliati-Dezza I, Brelsford V. et al Baby schema in human and animal faces induces cuteness perception and gaze allocation in children. Front Psychol 2014; 5: 411 DOI: 10.3389/fpsyg.2014.00411.
• 27 Luo LZ, Li H, Lee K. Are children's faces really more appealing than those of adults? Testing the baby schema hypothesis beyond infancy. J Exp Child Psychol 2011; 110: 115-124 DOI: 10.1016/j.jecp.2011.04.002.
• 28 Charness N, Boot WR. Aging and Information Technology Use. Curr Dir Psychol Sci 2009; 18: 253-258 DOI: 10.1111/j.1467-8721.2009.01647.x.
• 29 Ellis D, Allaire JC. Modeling Computer Interest in Older Adults: the Role of Age, Education, Computer Knowledge, and Computer Anxiety. Hum Factors 1999; 41: 345-355 DOI: 10.1518/001872099779610996.

Publication History

Received: 27 July 2020

Accepted: 13 January 2021

Article published online:
10 June 2021