PDF herunterladen
Yearb Med Inform 2013; 22(01): 78-85
DOI: 10.1055/s-0038-1638836
Original Article
Georg Thieme Verlag KG Stuttgart

From Usability Testing to Clinical Simulations: Bringing Context into the Design and Evaluation of Usable and Safe Health Information Technologies

Contribution of the IMIA Human Factors Engineering for Healthcare Informatics Working Group
A. Kushniruk
1   School of Health Information Science, University of Victoria, Victoria, Canada
2   Aalborg University, Department of Development and Planning, Aalborg, Denmark
,
C. Nohr
2   Aalborg University, Department of Development and Planning, Aalborg, Denmark
,
S. Jensen
2   Aalborg University, Department of Development and Planning, Aalborg, Denmark
,
E. M. Borycki
1   School of Health Information Science, University of Victoria, Victoria, Canada
› Institutsangaben
Weitere Informationen

Publikationsverlauf

Publikationsdatum:
05. März 2018 (online)

Auch verfügbar auf
   Lizenzen und Reprints

Summary

Objectives: The objective of this paper is to explore human factors approaches to understanding the use of health information technology (HIT) by extending usability engineering approaches to include analysis of the impact of clinical context through use of clinical simulations.

Methods: Methods discussed are considered on a continuum from traditional laboratory-based usability testing to clinical simulations. Clinical simulations can be conducted in a simulation laboratory and they can also be conducted in real-world settings. The clinical simulation approach attempts to bring the dimension of clinical context into stronger focus. This involves testing of systems with representative users doing representative tasks, in representative settings/environments.

Results: Application of methods where realistic clinical scenarios are used to drive the study of users interacting with systems under realistic conditions and settings can lead to identification of problems and issues with systems that may not be detected using traditional usability engineering methods. In conducting such studies, careful consideration is needed in creating ecologically valid test scenarios. The evidence obtained from such evaluation can be used to improve both the usability and safety of HIT. In addition, recent work has shown that clinical simulations, in particular those conducted in-situ, can lead to considerable benefits when compared to the costs of running such studies.

Conclusion: In order to bring context of use into the testing of HIT, clinical simulation, involving observing representative users carrying out tasks in representative settings, holds considerable promise.

Keywords

Ussability - clinical simulation - patient safety - technology-induced error - health information systems
 

  • References

  • 1 Kushniruk AW, Triola MM, Borycki EM, Stein B, Kannry JL. The relationship between usability problems and prescription errors when using a handheld application. Int J Med Inform 2005; 74 (7- 8) 519-26.
    CrossrefPubMedGoogle Scholar
  • 2 Koppel R, Metlay JP, Cohen A, Abaluck B, Localio AR, Kimmel SE. et al. Role of CPOE in facilitating medication errors. J Am Med Inform Assoc 2005; 293 (10) 1197-203.
    CrossrefPubMedGoogle Scholar
  • 3 Magrabi F, Ong M-S, Runciman W, Coiera E. An analysis of computer-related patient safety incidents to inform the development of a classification. J Am Med Inform Assoc 2010; 17: 663-70.
    CrossrefPubMedGoogle Scholar
  • 4 Magrabi F, Ong M, Runciman W, Coiera E. Using FDA reports to inform classification of HIT safety problems. J Am Med Inform Assoc 2012; 19: 45-53.
    CrossrefPubMedGoogle Scholar
  • 5 Samaranayake NR, Cheug STD, Chui WCM. Cheung BMY. Technology-related medication errors in a tertiary hospital: A 5-year analysis of reported medication incidents. Int J Med Inform 2012; 81: 828-33.
    CrossrefPubMedGoogle Scholar
  • 6 Carvalho CJ, Borycki EM, Kushniruk AK. Ensuring the safety of health information systems. Healthc Q 2009; 12: 49-54.
    CrossrefPubMedGoogle Scholar
  • 7 Rasmussen R, Kushniruk A. The long and twisting path: An efficiency evaluation of an electronic whiteboard system. Stud Health Technol Inform 2013; 183: 174-8.
    PubMedGoogle Scholar
  • 8 Lilholt LH, Pedersen SS, Madsen I, Nielsen PH, Boye N, Andersen SK, Nohr C. Development of methods for usability evaluations of EHR systems. Stud Health Technol Inform 2006; 124: 341-6.
    PubMedGoogle Scholar
  • 9 Borycki EM, Kushniruk AW, Kuwata S, Kannry J. Engineering the electronic health record for safety: A multi-video-based approach to diagnosing and preventing technology-induced error arising from usability problems. Stud Health Technol Inform 2011; 166: 197-205.
    PubMedGoogle Scholar
  • 10 Beuscart-Zephir MC, Pelayo S, Anceaux F, Meaux J, Degroisse M, Degoulet P. Impact of CPOE on doctor-nurse cooperation for the medication ordering and administration process. Int J Med Inform 2005; 74 (7) 629-41.
    CrossrefPubMedGoogle Scholar
  • 11 Carvalho CJ, Borycki EM, Kushniruk AK. Ensuring the safety of health information systems. Healthc Q 2009; 12: 49-54.
    CrossrefPubMedGoogle Scholar
  • 12 Kushniruk A, Turner P. A framework for user involvement and context in the design and development of safe e-Health systems. Stud Health Technol Inform 2012; 180: 353-7.
    PubMedGoogle Scholar
  • 13 Borycki EM, Lemieux-Charles L, Nagle L, Eysenbach G. Evaluating the impact of hybrid electronic-paper environments upon novice nurse information seeking. Methods Inf Med 2009; 48 (2) 137-43.
    Artikel in Thieme ConnectPubMedGoogle Scholar
  • 14 Pearce C, Shachak A, Kushniruk A, de Lusignan S. Usability: A critical dimension for assessing the quality of clinical systems. Inform Prim Care 2009; 17 (4) 195-8.
    PubMedGoogle Scholar
  • 15 Kushniruk AW, Patel VL. Cognitive and usability engineering methods for the evaluation of clinical information systems. J Biomed Inform 2004; 37 (1) 56-76.
    CrossrefPubMedGoogle Scholar
  • 16 Ammenwerth E, Hackl WO, Bintzer K, Christoffersen TEH, Jensen S, Lawton K. et al. Simulation studies for the evaluation of health information technologies: experiences and results. HIM J 2012; 41 (2) 14-21.
    CrossrefPubMedGoogle Scholar
  • 17 Borycki E, Kushniruk A, Anderson J, Anderson M. Designing and integrating clinical and computer-based simulations in health informatics: From real-world to virtual reality. In: Cakaj S. editor. Modeling Simulation and Optimization: Focus on Applications. Vukovar, Croatia: In-Tech; 2010. p. 31-52.
    Google Scholar
  • 18 Borycki EM, Keay L. Methods to assess the safety of health information systems. Healthc Q 2010; 13: 49-54.
    CrossrefPubMedGoogle Scholar
  • 19 Kushniruk AW, Borycki EM, Kuwata S, Watanabe H. Using a low-cost simulation for assessing the impact of a medication administration system on workflow. Stud Health Technol Inform 2008; 136: 567-72.
    PubMedGoogle Scholar
  • 20 Zheng K, Padman R, Johnson MP, Diamond HS. Understanding technology adoption in clincal care: Clinician adoption behaviour of a point-of-care reminder system. IntJ Med Inform 2005; 74 (7) 535-43.
    PubMedGoogle Scholar
  • 21 Kushniruk AW, Beuscart-Zephir MC, Grzes A, Borycki E, Watbled L, Kannry J. Increasing the safety of healthcare information systems through improved procurement: Toward a framework for selection of safe healthcare systems. Healthc Q 2010; 13: 53-8.
    CrossrefPubMedGoogle Scholar
  • 22 Marcilly R, Bernonville S, Riccioli C, Beuscart-Zephir MC. Patient safety-oriented usability testing: A pilot study. Stud Health Technol Inform 2012; 180: 368-72.
    PubMedGoogle Scholar
  • 23 Nielsen J, Mack R. Usability inspection methods. New York: John Wiley; 1994
    Google Scholar
  • 24 Sharp H, Preece J. Interaction design (3rd ed.). New York: Wiley; 2011
    Google Scholar
  • 25 Kushniruk AW, Borycki EM. Low-cost usability engineering. Healthc Q 2006; 9 (4) 98-100 102.
    PubMedGoogle Scholar
  • 26 Kushniruk AW, Borycki EM, Kannry J. Commercial versus in-situ usability testing of healthcare information systems: Towards “public” usability tesing in healthcare organizations. Stud Health Technol Inform 2013; 183: 157-61.
    PubMedGoogle Scholar
  • 27 Baylis T, Kushniruk AW, Borycki EM. Low-cost usability for health information systems: Is it worth the effort?. Stud Health Technol Inform 2012; 180: 363-7.
    PubMedGoogle Scholar
  • 28 Hall SA, Kushniruk AW, Borycki EM. Usability analysis of the tele-nursing call management software at HealthLink BC. Stud Health Technol Inform 2011; 164: 208-12.
    PubMedGoogle Scholar
  • 29 Li AC, Kannry JL, Kushniruk A, Chrimes D, McGinn TG, Edonyabo D, Mann DM. Integrating usability testing and think-aloud protocol analysis with "near live" clincal simulations in evaluting clinical decision support. Int J Med Inform 2012; 81 (11) 761-21.
    CrossrefPubMedGoogle Scholar
  • 30 Kushniruk AW, Borycki EM, Kuwata S, Kannry J. Emerging approaches to usability evaluation of health information systems: Towards in-situ analysis of complex healthcare systems and envioronments. Stud Health Technol Inform 2011; 169: 915-9.
    PubMedGoogle Scholar
  • 31 Rasmussen SL, Lyng KM, Jensen S. Achieving IT-supported standardized mursing documentation through participatory design. Stud Health Technol Inform 2012; 180: 1055-9.
    PubMedGoogle Scholar
  • 32 Ammenwerth E, Hackl WO, Jensen S, Lawton K, Riedmann D, Jung M. Impact evaluation of innovative technology: estimating the impact of the PSIP solutions. Stud Health Technol Inform 2011; 166: 227-33.
    PubMedGoogle Scholar
  • 33 Kjeldskov J, Skov MB, Stage J. Instant data analysis: conducting usability evaluations in a day. NordiCHI '04 Proceedings of the third Nordic conference on Human-computer interaction, ACM. 2004
    PubMedGoogle Scholar
  • 34 Jensen S, Kanstrup AM, Nohr C. User driven development and high fidelity testing. Proceedings Eleventh Danish HCI Research Symposium. 2011
    PubMedGoogle Scholar
  • 35 Jensen S, Vingtoft S, Nohr C. Benefits of a clinical planning and coordination module: A simulation study. Stud Health Technol Inform 2013; 183: 220-4.
    PubMedGoogle Scholar
  • 36 Lawton K, Bintzer K, Skjoet P, Jensen S. Lessons learned from conducting a high fidelity simulation test in health IT. Stud Health Technol Inform 2011; 166: 217-26.
    PubMedGoogle Scholar
  • 37 Interaction design. Ecological validity. [Internet]. Accessed from: http://www.interaction-design.org/encyclopedia/ecological_validity.html
    PubMedGoogle Scholar
  • 38 Elmes DG, Kantowitz BH, Roediger HL. Research methods in psychology. New York: Wadsworth Publishing Company; 2011
    Google Scholar
  • 39 Sanderson P. Designing and evaluating healthcare ICT innovation: A cognitive engineering view. Proceedings of the Third International Conference on Information Technology in Healthcare-Socio-technical Systems. Sydney: 28-30 August 2007
    Google Scholar
  • 40 Carayon P. Human factors and ergonomics in health care and patient safety (Second edition). New York: CRC Press; 2012
    Google Scholar
  • 41 Jaspers M. A comparison of usability methods for testing interactive health technologies: Methodological aspects and empirical evidence. Int J Med Inform 2009; 78: 340-53.
    CrossrefPubMedGoogle Scholar