Download PDF
Appl Clin Inform 2013; 04(03): 331-358
DOI: 10.4338/ACI-2013-04-RA-0024
Research Article
Schattauer GmbH

STARE-HI – Statement on Reporting of Evaluation Studies in Health Informatics

Explanation and Elaboration
J. Brender
1   Department of Health Science and Technology, Aalborg University, and V-CHI, Aalborg, Denmark
,
J. Talmon
2   School of Public Health and Primary Care – CAPHRI, Maastricht University, Maastricht, The Netherlands
,
N. de Keizer
3   Department of Medical Informatics, Academic Medical Center, Amsterdam, The Netherlands
,
P. Nykänen
4   School of Information Sciences, University of Tampere, Tampere, Finland
,
M. Rigby
5   School of Public Policy and Professional Practice, Keele University, Keele, United Kingdom
,
E. Ammenwerth
6   Institute of Medical Informatics, UMIT – University for Health Sciences, Medical Informatics and Technology, Hall in Tyrol, Austria
› Author Affiliations
Further Information

Publication History

received: 20 April 2013

accepted: 29 June 2013

Publication Date:
16 December 2017 (online)

Also available at
   Permissions and Reprints

Summary

Background: Improving the quality of reporting of evaluation studies in health informatics is an important requirement towards the vision of evidence-based health informatics. The STARE-HI – Statement on Reporting of Evaluation Studies in health informatics, published in 2009, provides guidelines on the elements to be contained in an evaluation study report.

Objectives: To elaborate on and provide a rationale for the principles of STARE-HI and to guide authors and readers of evaluation studies in health informatics by providing explanatory examples of reporting.

Methods: A group of methodologists, researchers and editors prepared the present elaboration of the STARE-HI statement and selected examples from the literature.

Results: The 35 STARE-HI items to be addressed in evaluation papers describing health informatics interventions are discussed one by one and each is extended with examples and elaborations. Conclusion: The STARE-HI statement and this elaboration document should be helpful resources to improve reporting of both quantitative and qualitative evaluation studies. Evaluation manuscripts adhering to the principles will enable readers of such papers to better place the studies in a proper context and judge their validity and generalizability, and thus in turn optimize the exploitation of the evidence contained therein.

Limitations: This paper is based on experiences of a group of editors, reviewers, authors of systematic reviews and readers of the scientific literature. The applicability of the details of these principles has to evolve as a function of their use in practice.

Keywords

Evaluation studie - guidelines - health information systems - informatics
 

  • References

  • 1 Altman DG, Schulz KF, Moher D, Egger M, Davidoff F, Elbourne D, Gotzsche PC, Lang T, Consort G. The revised CONSORT statement for reporting randomized trials: explanation and elaboration.. Ann Intern Med 2001; 8: 663-694.
    CrossrefPubMedGoogle Scholar
  • 2 Ammenwerth E, Brender J, Nykänen P, Prokosch H-U, Rigby M, Talmon J. Visions and strategies to improve evaluation of health information systems - reflections and lessons based on the HIS-EVAL workshop in Innsbruck.. Int J Med Inform 2004; 6: 479-491.
    CrossrefPubMedGoogle Scholar
  • 3 Ash J. Organizational factors that influence information technology diffusion in academic health sciences centers.. J Am Med Inform Assoc 1997; 2: 102-111.
    CrossrefPubMedGoogle Scholar
  • 4 Ash JS, Berg M, Coiera E. Some unintended consequences of information technology in health care: the nature of patient care information system-related errors.. J Am Med Inform Assoc 2004; 2: 104-112.
    PubMedGoogle Scholar
  • 5 Berner E, Moss J. Informatics Challenges for the Impending Patient Information Explosion.. J Am Med Inform Assoc 2005; 6: 614-617.
    CrossrefPubMedGoogle Scholar
  • 6 Berner ES, Houston TK, Ray MN, Allison JJ, Heudebert GR, Chatham WW, Kennedy Jr., JI, Glandon GL, Norton PA, Crawford MA, Maisiak RS. Improving ambulatory prescribing safety with a handheld decision support system: a randomized controlled trial.. J Am Med Inform Assoc 2006; 2: 171-179. doi: 10.1197/jamia. M1961.
    CrossrefPubMedGoogle Scholar
  • 7 Bloomrosen M, Starren J, Lorenzi NM, Ash JS, Patel VL, Shortliffe EH. Anticipating and addressing the unintended consequences of health IT and policy: a report from the AMIA 2009 Health Policy Meeting.. J Am Med Inform Assoc 2011; 1: 82-90. doi: 10.1136/jamia.2010.007567.
    CrossrefPubMedGoogle Scholar
  • 8 Brender J. Handbook of evaluation methods for health informatics.. Burlington, MA: Elsevier Academic Press; 2006
    Google Scholar
  • 9 Brender J, Talmon J. On using references as evidence.. Methods of information in medicine 2009; 6: 503-507. doi: 10.3414/ME09-03-0001.
    Article in Thieme ConnectPubMedGoogle Scholar
  • 10 Callen JL, Westbrook JI, Braithwaite J. The effect of physicians’ long-term use of CPOE on their test management work practices.. J Am Med Inform Assoc 2006; 6: 643-652. doi: 10.1197/jamia. M2152.
    CrossrefPubMedGoogle Scholar
  • 11 Chertow GM, Lee J, Kuperman GJ, Burdick E, Horsky J, Seger DL, Lee R, Mekala A, Song J, Komaroff AL, Bates DW. Guided medication dosing for inpatients with renal insufficiency.. Jama 2001; 22: 2839-2844.
    CrossrefPubMedGoogle Scholar
  • 12 Coolican H. Introduction to Research Methods and Statistics in Psychology.. Ldon: Hodder & Stoughton; 1999
    Google Scholar
  • 13 Eminovic N, de Keizer NF, Wyatt JC, ter Riet G, Peek N, van Weert HC, Bruijnzeel-Koomen CA, Bindels PJ. Teledermatologic consultation and reduction in referrals to dermatologists: a cluster randomized controlled trial.. Arch Dermatol 2009; 5: 558-564. doi: 10.1001/archdermatol.2009.44.
    PubMedGoogle Scholar
  • 14 Fraenkel DJ, Cowie M, Daley P. Quality benefits of an intensive care clinical information system.. Crit Care Med 2003; 1: 120-125.
    CrossrefPubMedGoogle Scholar
  • 15 Friedman C, Wyatt JC. Evaluation Methods in Medical Informatics.. New York: Springer; 2006
    Google Scholar
  • 16 Gandhi TK, Weingart SN, Seger AC, Borus J, Burdick E, Poon EG, Leape LL, Bates DW. Outpatient prescribing errors and the impact of computerized prescribing.. J Gen Intern Med 2005; 9: 837-841.
    CrossrefPubMedGoogle Scholar
  • 17 Goud R, de Keizer NF, ter Riet G, Wyatt JC, Hasman A, Hellemans IM, Peek N. Effect of guideline based computerised decision support on decision making of multidisciplinary teams: cluster randomised trial in cardiac rehabilitation.. BMJ. 2009 b1440. doi: 10.1136/bmj.b1440.
    PubMedGoogle Scholar
  • 18 Han YY, Carcillo JA, Venkataraman ST, Clark RS, Watson RS, Nguyen TC, Bayir H, Orr RA. Unexpected increased mortality after implementation of a commercially sold computerized physician order entry system.. Pediatrics 2005; 6: 1506-1512.
    CrossrefPubMedGoogle Scholar
  • 19 Hansson S. Values in pure and applied science.. Found Sci 2007: 257-268.
    PubMedGoogle Scholar
  • 20 Harris A, McGregor J, Perencevich E, Furuno J, Zhu J, Peterson D, Finkelstein J. The use and interpretation of quasi-experimental studies in medical informatics.. J Am Med Inform Assoc 2006; 1: 16-23.
    CrossrefPubMedGoogle Scholar
  • 21 Haux R. Medical informatics: past, present, future.. Int J Med Inform 2010; 9: 599-610. doi: S1386-5056(10)00114-0 [pii] 10.1016/j.ijmedinf.2010.06.003.
    CrossrefPubMedGoogle Scholar
  • 22 ICMJE.. Uniform Requirements for Manuscripts Submitted to Biomedical Journals: Writing and Editing for Biomedical Publications.. 2012, March-28. http://www.icmje.org/index.html.
    PubMedGoogle Scholar
  • 23 Kaplan B, Lundsgaarde HP. Toward an evaluation of a clinical imaging system: identifying benefits.. Methods of Information in Medicine 1996; 3: 221-229.
    Article in Thieme ConnectPubMedGoogle Scholar
  • 24 Kastner M, Li J, Lottridge D, Marquez C, Newton D, Straus SE. Development of a prototype clinical decision support tool for osteoporosis disease management: a qualitative study of focus groups.. BMC Med Inform Decis Mak 2010; 40. doi: 10.1186/1472-6947-10-40.
    CrossrefPubMedGoogle Scholar
  • 25 Koppel R, Metlay J, Cohen A, Abaluck B, Localio A, SE K, Strom B. Role of computerized physician order entry systems in facilitating medication errors.. JAMA 2005; 10: 1197-2003.
    CrossrefPubMedGoogle Scholar
  • 26 Kushniruk AW, Triola MM, Borycki EM, Stein B, Kannry JL. Technology induced error and usability: the relationship between usability problems and prescription errors when using a handheld application.. Int J Med Inform 2005; 7-8 519-526. doi: S1386-5056(05)00011-0 [pii] 10.1016/j.ijmedinf.2005.01.003.
    PubMedGoogle Scholar
  • 27 Layman E, Watzlaf V. Health Informatics Research Methods: Principles and Practice.. Chicago, IL: AHIMA; 2009
    Google Scholar
  • 28 Moher D, Hopewell S, Schulz KF, Montori V, Gotzsche PC, Devereaux PJ, Elbourne D, Egger M, Altman DG. CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials.. BMJ 2010; c869. doi: 10.1136/bmj.c869.
    CrossrefPubMedGoogle Scholar
  • 29 NLM.. MeSH. Medical Subject Headings.. http://www.nlm.nih.gov/mesh/MBrowser.html.
    PubMedGoogle Scholar
  • 30 Nykanen P, Brender J, Talmon J, de Keizer N, Rigby M, Beuscart-Zephir MC, Ammenwerth E. Guideline for Good Evaluation Practice in Health Informatics (GEP-HI).. Int J Med Inform 2011: 815-827. doi: S1386-5056(11)00168-7 [pii] 10.1016/j.ijmedinf.2011.08.004.
    PubMedGoogle Scholar
  • 31 Palm JM, Colombet I, Sicotte C, Degoulet P. Determinants of user satisfaction with a Clinical Information System.. AMIA Annu Symp Proc 2006: 614-618.
    PubMedGoogle Scholar
  • 32 Peterson JF, Kuperman GJ, Shek C, Patel M, Avorn J, Bates DW. Guided prescription of psychotropic medications for geriatric inpatients.. Arch Intern Med 2005; 7: 802-807.
    CrossrefPubMedGoogle Scholar
  • 33 Peute LW, Jaspers MW. The significance of a usability evaluation of an emerging laboratory order entry system.. Int J Med Inform 2007; 2-3 : 157-168. doi: 10.1016/j.ijmedinf.2006.06.003.
    PubMedGoogle Scholar
  • 34 Potts AL, Barr FE, Gregory DF, Wright L, Patel NR. Computerized physician order entry and medication errors in a pediatric critical care unit.. Pediatrics 2004; 1 Pt 1 59-63.
    CrossrefPubMedGoogle Scholar
  • 35 Rigby M. Evaluation - the cinderella science of health IMIA Yearbook of Medical Informatics. 2006: 114-120.
    PubMedGoogle Scholar
  • 36 Strom BL, Schinnar R, Aberra F, Bilker W, Hennessy S, Leonard CE, Pifer E. Unintended effects of a computerized physician order entry nearly hard-stop alert to prevent a drug interaction: a randomized controlled trial.. Arch Intern Med 2010; 17: 1578-1583. doi: 10.1001/archinternmed.2010.324.
    PubMedGoogle Scholar
  • 37 Talmon J, Ammenwerth E, Geven T. The quality of reporting of health informatics evaluation studies: a pilot study.. Stud Health Technol Inform. 2007 ; Pt 1: 193-197.
    PubMedGoogle Scholar
  • 38 Talmon J, Ammenwerth A, Brender J, de Keizer N, Nykänen P, Rigby M. STARE-HI - Statement on Reporting of Evaluation Studies in Health Informatics.. Int J Med Inform 2009; 1: 1-9.
    CrossrefPubMedGoogle Scholar
  • 39 Tufte E. The Visual Display of Quantitative Information.. Cheshire (Connecticut): Graphics Press; 2001
    Google Scholar
  • 40 van den Brink JL, Moorman PW, de Boer MF, Pruyn JF, Verwoerd CD, van Bemmel JH. Involving the patient: a prospective study on use, appreciation and effectiveness of an information system in head and neck cancer care.. Int J Med Inform 2005; 10: 839-849. doi: 10.1016/j.ijmedinf.2005.03.021.
    CrossrefPubMedGoogle Scholar
  • 41 Vandenbroucke JP, von Elm E, Altman DG, Gotzsche PC, Mulrow CD, Pocock SJ, Poole C, Schlesselman JJ, Egger M. Initiative S. Strengthening the Reporting of Observational Studies in Epidemiology (STROBE): explanation and elaboration.. Ann Intern Med 2007; 8: W163-W194.
    PubMedGoogle Scholar
  • 42 Vervloet M, van Dijk L, Santen-Reestman J, van Vlijmen B, van Wingerden P, Bouvy ML, de Bakker DH. SMS reminders improve adherence to oral medication in type 2 diabetes patients who are real time electronically monitored.. Int J Med Inform 2012; 9: 594-604. doi: 10.1016/j.ijmedinf.2012.05.005.
    CrossrefPubMedGoogle Scholar
  • 43 Winter A, Haux R, Ammenwerth E, Brigl B, Hellrung N, Jahn F. Health Information Systems: Architectures and Strategies.. New York, Berlin, Heidelberg: Springer-Verlag; 2011
    CrossrefGoogle Scholar