The Development of Heuristics for Evaluation of Dashboard Visualizations
Dawn Dowding
1
Division of Nursing, Midwifery and Social Work, School of Health Sciences, University of Manchester, Manchester, United Kingdom
,
Jacqueline A. Merrill
2
Department of Biomedical Informatics, Columbia University, New York, New York, United States
3
School of Nursing, Columbia University, New York, New York, United States
› InstitutsangabenFunding The research reported here was supported by the Agency for Healthcare Research and Quality (U.S.) under award number R21HS023855. The content is solely the responsibility of the authors and does not necessarily represent the official viess of the Agency for Healthcare Research and Quality.
Background Heuristic evaluation is used in human–computer interaction studies to assess the usability of information systems. Nielsen's widely used heuristics, first developed in 1990, are appropriate for general usability but do not specifically address usability in systems that produce information visualizations.
Objective This article develops a heuristic evaluation checklist that can be used to evaluate systems that produce information visualizations. Principles from Nielsen's heuristics were combined with heuristic principles developed by prior researchers specifically to evaluate information visualization.
Methods We used nominal group technique to determine an appropriate final set. The combined existing usability principles and associated factors were distributed via email to a group of 12 informatics experts from a range of health care disciplines. Respondents were asked to rate each factor on its importance as an evaluation heuristic for visualization systems on a scale from 1 (definitely don't include) to 10 (definitely include). The distribution of scores for each item were calculated. A median score of ≥8 represented consensus for inclusion in the final checklist.
Results Ten of 12 experts responded with rankings and written comments. The final checklist consists of 10 usability principles (7 general and 3 specific to information visualization) substantiated by 49 usability factors. Three nursing informatics experts then used the checklist to evaluate a vital sign dashboard developed for home care nurses, using a task list designed to explore the full functionality of the dashboard. The experts used the checklist without difficulty, and indicated that it covered all major usability problems encountered during task completion.
Conclusion The growing capacity to generate and electronically process health data suggests that data visualization will be increasingly important. A checklist of usability heuristics for evaluating information visualization systems can contribute to assuring high quality in electronic data systems developed for health care.
Keywords
requirements analysis and design -
clinical decision support -
home health -
dashboard -
methodologies
Protection of Human and Animal Subjects
The study protocol was reviewed and approved by the Institutional Review Boards at Columbia University and the Visiting Nurse Service of New York.
1
Andrews K.
Evaluation comes in many guises. In: Proceedings of the 2008 AVI workshop on BEyond time and errors: novel evaluation methods for information visualization (BELIV, 2008); Florence, Italy. New York: ACM; 2008
2
Hermawati S,
Lawson G.
Establishing usability heuristics for heuristics evaluation in a specific domain: is there a consensus?. Appl Ergon 2016; 56: 34-51
4
Nielsen J,
Molich R.
Heuristic evaluation of user interfaces. In: CHI '90 Proceedings of the SIGCHI Conference on Human Factors in Computing Systems Seattle, Washington. New York: ACM; 1990 :249–256
6
Tarrell A,
Forsell C,
Fruhling AL,
Grinstein G,
Borgo R,
Scholtz J.
Toward Visualization-Specific Heuristic Evaluation. In: BELIV '14 Proceedings of the Fifth Workshop on Beyond Time and Errors: Novel Evaluation Methods for Visualization Paris, France. New York: ACM; 2014 :110–117
7
Väätäjä H,
Varsaluoma J,
Heimonen T.
, et al. Information visualization heuristics in practical expert evaluation. In: BELIV '16 Proceedings of the Sixth Workshop on Beyond Time and Errors on Novel Evaluation Methods for Visualization Baltimore, MD. New York: ACM; 2016 :36–43
8
Gonzalez-Holland E,
Whitmer D,
Moralez L,
Mouloua M.
Examination of the use of Nielsen's 10 usability heuristics & outlooks for the future. Proc Hum Factors Ergon Soc Annu Meet 2017; 61 (01) 1472-1475
9
Forsell C,
Johansson J.
An heuristic set for evaluation in information visualization. In: AVI '10 Proceedings of the International Conference on Advanced Visual Interfaces Roma, Italy. New York: ACM; 2010 :199–206
10
Zuk T,
Schlesier L,
Neumann P,
Hancock MS,
Carpendale S.
Heuristics for information visualization evaluation. In: BELIV '06 Proceedings of the 2006 AVI workshop on BEyond time and errors: novel evaluation methods for information visualization Venice, Italy. New York: ACM; 2006 :1–6
11
Freitas C,
Luzzardi P,
Cava R,
Winckler M,
Pimenta MS,
Nedel L.
Evaluating usability of information visualization techniques. In: Proceedings of the 5th Symposium on Human Factors in Computer Systems, IHC 2002; Brazil. Brazil: Brazilian Computer Society; 2002 :40–51
12
Delbecq AL,
Van de Ven AH,
Gustafson DH.
Group Techniques for Program Planning: A Guide to Nominal Group and Delphi Processes. Glenview, IL: Scott Foresman; 1975
14
Amar R,
Stasko J.
A knowledge task-based framework for design and evaluation of information visualizations. In: Proceedings of IEEE Symposium on Information Visualization, 2004 INFOVIS 2004; Austin, TX, USA. Washington, DC: IEEE Computer Society; 2004
16
Shneiderman B.
The eyes have it: a task by data type taxonomy for information visualizations. In: Proceedings of the IEEE Symposium on Visual Languages; Boulder, CO, USA. Washington, DC: IEEE Computer Society Press; 1996 :336–343
17
Nielsen J.
Enhancing the explanatory power of usability heuristics. In: CHI '94 Proceedings of the SIGCHI Conference on Human Factors in Computing Systems: April 24–28, 1994; Boston, MA. New York, NY: ACM; 1994 :152–158
18
Zuk T,
Carpendale S.
Theoretical analysis of uncertainty visualizations. Proc. SPIE 6060, Visualization and Data Analysis 2006, 606007. Doi: 10.1117/12.643631
20
Dowding DW,
Russell D,
Onorato N,
Merrill JA.
Technology solutions to support care continuity in home care: a focus group study. J Healthc Qual 2017
21
Docherty SL,
Vorderstrasse A,
Brandon D,
Johnson C.
Visualization of multidimensional data in nursing science. West J Nurs Res 2016; 39 (01) 112-126
22
Schumacher R,
Lowry S.
NIST Guide to the Processes Approach for Improving the Usability of Electronic Health Records. Gaithersburg, MD: National Institute of Standards and Technology; 2010
23
Rind A,
Wang T,
Aigner W.
, et al. Interactive information visualization to explore and query electronic health records. Foundations Trends Human Comp Interact 2013; 5 (03) 207-298
24
Wang TD,
Wongsuphasawat K,
Plaisant C,
Shneiderman B.
Extracting insights from electronic health records: case studies, a visual analytics process model, and design recommendations. J Med Syst 2011; 35 (05) 1135-1152
25
West VL,
Borland D,
Hammond WE.
Innovative information visualization of electronic health record data: a systematic review. J Am Med Inform Assoc 2015; 22 (02) 330-339
27 National Library of Medicine (U.S.). Board of Regents: a platform for biomedical discovery and data-powered health: National Library of Medicine strategic plan 2017–202. In: NIH Publication. Bethesda, MD: National Institutes of Health, National Library of Medicine; 2017
