Heuristic Evaluation of a Top-Rated Diabetes Self-Management App

Abstract

Objective The purpose of this study was to evaluate the usability of a top-rated diabetes app. Such apps are intended to markedly support the achievement of optimal health and financial outcomes by providing patients with substantive and continual support for self-management of their disease between periodic clinician visits. Poor usability can deter use which is especially concerning in patients with diabetes due to prevalence of the disease and impact of self-management on long-term prognosis.

Methods A diabetes app was selected due to the prevalence and seriousness of the disease. A heuristic evaluation was then performed to collect and analyze data on the usability of the app based on Nielsen's heuristics. Pareto analysis was used to illustrate the contribution of each type of heuristic violation, augmented by a stacked bar chart illuminating associated severity.

Results There were 51 heuristic violations on the opening screen, violating 6 of Nielsen's 10 heuristics. Pareto analysis revealed 29 (57%) of the heuristic violations involved a match between system and real world and 8 (16%) aesthetic and minimalist design. Severity ratings ranged from 1.0 to 4.0 (mean: 3.01) with 80% comprising a major usability problem and 6% a usability catastrophe.

Conclusion Studies show that people with diabetes are more likely to receive greater benefit from a diabetes app if they are easy to use. The number and severity of heuristic violations in this study suggest that the commercialization of mobile health apps may play a factor in bypassing experts in clinical informatics during the design phase of development. Usability and associated benefits received from mobile health apps can be enhanced by debugging the user interface of identified heuristic violations during design. Waiting to correct ongoing usability issues while apps are in production can result in patients disengaging from use of digital health tools engendering poorer outcomes.

Keywords

usability testing - mobile apps - user needs - interface design

Protection of Human and Animal Subjects

The study was performed in compliance with the World Medical Association Declaration of Helsinki on Ethical Principles for Medical Research Involving Human Subjects, and was deemed exempt by Texas Christian University, Institutional Review Board Chair.

Publication History

Received: 21 June 2021

Accepted: 13 September 2021

Article published online:
03 November 2021

• References

• 1 Saeedi P, Salpea P, Karuranga S. et al. Mortality attributable to diabetes in 20-79 years old adults, 2019 estimates: results from the International Diabetes Federation Diabetes Atlas, 9^th edition. Diabetes Res Clin Pract 2020; 162: 108086
• 2 American Diabetes Association. 6. Glycemic targets: standards of medical care in diabetes–2020. Diabetes Care 2020; 43 (Suppl. 01) S66-S76
• 3 Hermanns N, Ehrmann D, Finke-Groene K, Kulzer B. Trends in diabetes self-management education: where are we coming from and where are we going? A narrative review. Diabet Med 2020; 37 (03) 436-447
• 4 Fleming GA, Petrie JR, Bergenstal RM, Holl RW, Peters AL, Heinemann L. Diabetes digital app technology: benefits, challenges, and recommendations. A consensus report by the European Association for the Study of Diabetes (EASD) and the American Diabetes Association (ADA) Diabetes Technology Working Group. Diabetes Care 2020; 43 (01) 250-260
• 5 Bankmycell.com. How many smartphones are in the world?. 2021 Accessed June 19, 2021 at: https://www.bankmycell.com/blog/how-many-phones-are-in-the-world
• 6 National Institute of Standards and Technology. Health Information Technology. Usability. 2011. Accessed June 19, 2021 at: https://www.nist.gov/system/files/documents/2017/05/09/Usability-Fact-Sheet-14FEB11.pdf
• 7 Nielsen J. Finding usability problems through heuristic evaluation. Paper presented at: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems—CHI '92. 1992: 373-380
• 8 van Haasteren A, Gille F, Fadda M, Vayena E. Development of the mHealth App Trustworthiness checklist. Digit Health 2019; 5: 2055207619886463
• 9 Baptista S, Trawley S, Pouwer F, Oldenburg B, Wadley G, Speight J. What do adults with type 2 diabetes want from the “Perfect” app? Results from the Second Diabetes MILES: Australia (MILLES-2) study. Diabetes Technol Ther 2019; 21 (07) 393-399
• 10 Adu MD, Malabu UH, Malau-Aduli AEO, Malau-Aduli BS. Users' preferences and design recommendations to promote engagements with mobile apps for diabetes self-management: Multi-national perspectives. PLoS One 2018; 13 (12) e0208942
• 11 Lunde P, Nilsson BB, Bergland A, Kværner KJ, Bye A. The effectiveness of smartphone apps for lifestyle improvement in noncommunicable diseases: systematic review and meta-analyses. J Med Internet Res 2018; 20 (05) e162
• 12 Nielsen J, Molich R. Heuristic evaluation of user interfaces. Paper presented at: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems - CHI '90. 1990: 249-256
• 13 Singh K, Drouin K, Newmark LP. et al. Many mobile health apps target high-need, high-cost populations, but gaps Remain. Health Aff (Millwood) 2016; 35 (12) 2310-2318
• 14 Rowland SP, Fitzgerald JE, Holme T, Powell J, McGregor A. What is the clinical value of mHealth for patients?. NPJ Digit Med 2020; 3: 4
• 15 Kasperbauer TJ, Wright DE. Expanded FDA regulation of health and wellness apps. Bioethics 2020; 34 (03) 235-241
• 16 Cohen AB, Mathews SC, Dorsey ER, Bates DW, Safavi K. Direct-to-consumer digital health. Lancet Digit Health 2020; 2 (04) e163-e165
• 17 Howlett V, Snyder C, Tamler H, Tullis TS, Wilson C. What the best usability specialists are made of. Paper presented at: CHI EA '02: CHI '02 Extended Abstracts on Human Factors in Computing Systems - CHI '02. 2002: 706-702
• 18 Nielsen J. How to conduct a heuristic evaluation.1994. Accessed June 19, 2021 at: https://www.nngroup.com/articles/how-to-conduct-a-heuristic-evaluation/
• 19 Molich R, Nielsen J. Improving a human-computer dialogue. Commun ACM 1990; 33 (03) 338-348
• 20 Nielsen J. 10 usability heuristics for user interface desiring. 2020. Accessed June 19, 2021 at: https://www.nngroup.com/articles/ten-usability-heuristics/
• 21 Nielsen J. Severity ratings for usability problems. 1994. Accessed July 27, 2021 at: https://www.nngroup.com/articles/how-to-rate-the-severity-of-usability-problems/
• 22 Wilkinson L. Revising the Pareto chart. Am Stat 2006; 60 (04) 332-334
• 23 Streit M, Gehlenborg N. Bar charts and box plots. Nat Methods 2014; 11 (02) 117
• 24 Kaley A. Match between the system and the real world: the 2^nd usability heuristic explained. 2018. NN/g Nielsen Norman Group. Accessed June 19, 2021 at: https://www.nngroup.com/articles/match-system-real-world/
• 25 Indratmo I, Howorko L, Boedianto JM, Daniel B. The efficacy of stacked bar charts in supporting single-attribute and overall-attribute comparisons. Visual Informatics 2018; 2: 155-165
• 26 Fu HNC, Rizvi RF, Wyman JF, Adam TJ. Usability evaluation of four top-rated commercially available diabetes app for adults with type 2 diabetes. Comput Inform Nurs 2020; 38 (06) 274-280
• 27 AlBesher A, Stone RT. The correlation between users' star rating and usability on mobile applications. International Scholarly and Scientific Research & Innovation 2017; 11 (05) 530-533
• 28 Magrabi F, Habli I, Sujan M. et al. Why is it so difficult to govern mobile apps in healthcare?. BMJ Health Care Inform 2019; 26 (01) e100006