Download PDF
Appl Clin Inform 2013; 04(03): 331-358
DOI: 10.4338/ACI-2013-04-RA-0024
Research Article
Schattauer GmbH

STARE-HI – Statement on Reporting of Evaluation Studies in Health Informatics

Explanation and Elaboration
J. Brender
1   Department of Health Science and Technology, Aalborg University, and V-CHI, Aalborg, Denmark
,
J. Talmon
2   School of Public Health and Primary Care – CAPHRI, Maastricht University, Maastricht, The Netherlands
,
N. de Keizer
3   Department of Medical Informatics, Academic Medical Center, Amsterdam, The Netherlands
,
P. Nykänen
4   School of Information Sciences, University of Tampere, Tampere, Finland
,
M. Rigby
5   School of Public Policy and Professional Practice, Keele University, Keele, United Kingdom
,
E. Ammenwerth
6   Institute of Medical Informatics, UMIT – University for Health Sciences, Medical Informatics and Technology, Hall in Tyrol, Austria
› Author Affiliations
Further Information

Publication History

received: 20 April 2013

accepted: 29 June 2013

Publication Date:
16 December 2017 (online)

Also available at
   Permissions and Reprints

Summary

Background: Improving the quality of reporting of evaluation studies in health informatics is an important requirement towards the vision of evidence-based health informatics. The STARE-HI – Statement on Reporting of Evaluation Studies in health informatics, published in 2009, provides guidelines on the elements to be contained in an evaluation study report.

Objectives: To elaborate on and provide a rationale for the principles of STARE-HI and to guide authors and readers of evaluation studies in health informatics by providing explanatory examples of reporting.

Methods: A group of methodologists, researchers and editors prepared the present elaboration of the STARE-HI statement and selected examples from the literature.

Results: The 35 STARE-HI items to be addressed in evaluation papers describing health informatics interventions are discussed one by one and each is extended with examples and elaborations. Conclusion: The STARE-HI statement and this elaboration document should be helpful resources to improve reporting of both quantitative and qualitative evaluation studies. Evaluation manuscripts adhering to the principles will enable readers of such papers to better place the studies in a proper context and judge their validity and generalizability, and thus in turn optimize the exploitation of the evidence contained therein.

Limitations: This paper is based on experiences of a group of editors, reviewers, authors of systematic reviews and readers of the scientific literature. The applicability of the details of these principles has to evolve as a function of their use in practice.

Keywords

Evaluation studie - guidelines - health information systems - informatics
 

  • References

  • 1 Altman DG, Schulz KF, Moher D, Egger M, Davidoff F, Elbourne D, Gotzsche PC, Lang T, Consort G. The revised CONSORT statement for reporting randomized trials: explanation and elaboration.. Ann Intern Med 2001; 8: 663-694.
    CrossrefPubMedSearch in Google Scholar
  • 2 Ammenwerth E, Brender J, Nykänen P, Prokosch H-U, Rigby M, Talmon J. Visions and strategies to improve evaluation of health information systems - reflections and lessons based on the HIS-EVAL workshop in Innsbruck.. Int J Med Inform 2004; 6: 479-491.
    CrossrefPubMedSearch in Google Scholar
  • 3 Ash J. Organizational factors that influence information technology diffusion in academic health sciences centers.. J Am Med Inform Assoc 1997; 2: 102-111.
    CrossrefPubMedSearch in Google Scholar
  • 4 Ash JS, Berg M, Coiera E. Some unintended consequences of information technology in health care: the nature of patient care information system-related errors.. J Am Med Inform Assoc 2004; 2: 104-112.
    PubMedSearch in Google Scholar
  • 5 Berner E, Moss J. Informatics Challenges for the Impending Patient Information Explosion.. J Am Med Inform Assoc 2005; 6: 614-617.
    CrossrefPubMedSearch in Google Scholar
  • 6 Berner ES, Houston TK, Ray MN, Allison JJ, Heudebert GR, Chatham WW, Kennedy Jr., JI, Glandon GL, Norton PA, Crawford MA, Maisiak RS. Improving ambulatory prescribing safety with a handheld decision support system: a randomized controlled trial.. J Am Med Inform Assoc 2006; 2: 171-179. doi: 10.1197/jamia. M1961.
    CrossrefPubMedSearch in Google Scholar
  • 7 Bloomrosen M, Starren J, Lorenzi NM, Ash JS, Patel VL, Shortliffe EH. Anticipating and addressing the unintended consequences of health IT and policy: a report from the AMIA 2009 Health Policy Meeting.. J Am Med Inform Assoc 2011; 1: 82-90. doi: 10.1136/jamia.2010.007567.
    CrossrefPubMedSearch in Google Scholar
  • 8 Brender J. Handbook of evaluation methods for health informatics.. Burlington, MA: Elsevier Academic Press; 2006
    Search in Google Scholar
  • 9 Brender J, Talmon J. On using references as evidence.. Methods of information in medicine 2009; 6: 503-507. doi: 10.3414/ME09-03-0001.
    Thieme ConnectPubMedSearch in Google Scholar
  • 10 Callen JL, Westbrook JI, Braithwaite J. The effect of physicians’ long-term use of CPOE on their test management work practices.. J Am Med Inform Assoc 2006; 6: 643-652. doi: 10.1197/jamia. M2152.
    CrossrefPubMedSearch in Google Scholar
  • 11 Chertow GM, Lee J, Kuperman GJ, Burdick E, Horsky J, Seger DL, Lee R, Mekala A, Song J, Komaroff AL, Bates DW. Guided medication dosing for inpatients with renal insufficiency.. Jama 2001; 22: 2839-2844.
    CrossrefPubMedSearch in Google Scholar
  • 12 Coolican H. Introduction to Research Methods and Statistics in Psychology.. Ldon: Hodder & Stoughton; 1999
    Search in Google Scholar
  • 13 Eminovic N, de Keizer NF, Wyatt JC, ter Riet G, Peek N, van Weert HC, Bruijnzeel-Koomen CA, Bindels PJ. Teledermatologic consultation and reduction in referrals to dermatologists: a cluster randomized controlled trial.. Arch Dermatol 2009; 5: 558-564. doi: 10.1001/archdermatol.2009.44.
    PubMedSearch in Google Scholar
  • 14 Fraenkel DJ, Cowie M, Daley P. Quality benefits of an intensive care clinical information system.. Crit Care Med 2003; 1: 120-125.
    CrossrefPubMedSearch in Google Scholar
  • 15 Friedman C, Wyatt JC. Evaluation Methods in Medical Informatics.. New York: Springer; 2006
    Search in Google Scholar
  • 16 Gandhi TK, Weingart SN, Seger AC, Borus J, Burdick E, Poon EG, Leape LL, Bates DW. Outpatient prescribing errors and the impact of computerized prescribing.. J Gen Intern Med 2005; 9: 837-841.
    CrossrefPubMedSearch in Google Scholar
  • 17 Goud R, de Keizer NF, ter Riet G, Wyatt JC, Hasman A, Hellemans IM, Peek N. Effect of guideline based computerised decision support on decision making of multidisciplinary teams: cluster randomised trial in cardiac rehabilitation.. BMJ. 2009 b1440. doi: 10.1136/bmj.b1440.
    PubMedSearch in Google Scholar
  • 18 Han YY, Carcillo JA, Venkataraman ST, Clark RS, Watson RS, Nguyen TC, Bayir H, Orr RA. Unexpected increased mortality after implementation of a commercially sold computerized physician order entry system.. Pediatrics 2005; 6: 1506-1512.
    CrossrefPubMedSearch in Google Scholar
  • 19 Hansson S. Values in pure and applied science.. Found Sci 2007: 257-268.
    PubMed
  • 20 Harris A, McGregor J, Perencevich E, Furuno J, Zhu J, Peterson D, Finkelstein J. The use and interpretation of quasi-experimental studies in medical informatics.. J Am Med Inform Assoc 2006; 1: 16-23.
    CrossrefPubMedSearch in Google Scholar
  • 21 Haux R. Medical informatics: past, present, future.. Int J Med Inform 2010; 9: 599-610. doi: S1386-5056(10)00114-0 [pii] 10.1016/j.ijmedinf.2010.06.003.
    CrossrefPubMedSearch in Google Scholar
  • 22 ICMJE.. Uniform Requirements for Manuscripts Submitted to Biomedical Journals: Writing and Editing for Biomedical Publications.. 2012, March-28. http://www.icmje.org/index.html.
    PubMed
  • 23 Kaplan B, Lundsgaarde HP. Toward an evaluation of a clinical imaging system: identifying benefits.. Methods of Information in Medicine 1996; 3: 221-229.
    Thieme ConnectPubMedSearch in Google Scholar
  • 24 Kastner M, Li J, Lottridge D, Marquez C, Newton D, Straus SE. Development of a prototype clinical decision support tool for osteoporosis disease management: a qualitative study of focus groups.. BMC Med Inform Decis Mak 2010; 40. doi: 10.1186/1472-6947-10-40.
    CrossrefPubMedSearch in Google Scholar
  • 25 Koppel R, Metlay J, Cohen A, Abaluck B, Localio A, SE K, Strom B. Role of computerized physician order entry systems in facilitating medication errors.. JAMA 2005; 10: 1197-2003.
    CrossrefPubMedSearch in Google Scholar
  • 26 Kushniruk AW, Triola MM, Borycki EM, Stein B, Kannry JL. Technology induced error and usability: the relationship between usability problems and prescription errors when using a handheld application.. Int J Med Inform 2005; 7-8 519-526. doi: S1386-5056(05)00011-0 [pii] 10.1016/j.ijmedinf.2005.01.003.
    PubMedSearch in Google Scholar
  • 27 Layman E, Watzlaf V. Health Informatics Research Methods: Principles and Practice.. Chicago, IL: AHIMA; 2009
    Search in Google Scholar
  • 28 Moher D, Hopewell S, Schulz KF, Montori V, Gotzsche PC, Devereaux PJ, Elbourne D, Egger M, Altman DG. CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials.. BMJ 2010; c869. doi: 10.1136/bmj.c869.
    CrossrefPubMedSearch in Google Scholar
  • 29 NLM.. MeSH. Medical Subject Headings.. http://www.nlm.nih.gov/mesh/MBrowser.html.
    PubMed
  • 30 Nykanen P, Brender J, Talmon J, de Keizer N, Rigby M, Beuscart-Zephir MC, Ammenwerth E. Guideline for Good Evaluation Practice in Health Informatics (GEP-HI).. Int J Med Inform 2011: 815-827. doi: S1386-5056(11)00168-7 [pii] 10.1016/j.ijmedinf.2011.08.004.
    PubMed
  • 31 Palm JM, Colombet I, Sicotte C, Degoulet P. Determinants of user satisfaction with a Clinical Information System.. AMIA Annu Symp Proc 2006: 614-618.
    PubMed
  • 32 Peterson JF, Kuperman GJ, Shek C, Patel M, Avorn J, Bates DW. Guided prescription of psychotropic medications for geriatric inpatients.. Arch Intern Med 2005; 7: 802-807.
    CrossrefPubMedSearch in Google Scholar
  • 33 Peute LW, Jaspers MW. The significance of a usability evaluation of an emerging laboratory order entry system.. Int J Med Inform 2007; 2-3 : 157-168. doi: 10.1016/j.ijmedinf.2006.06.003.
    PubMedSearch in Google Scholar
  • 34 Potts AL, Barr FE, Gregory DF, Wright L, Patel NR. Computerized physician order entry and medication errors in a pediatric critical care unit.. Pediatrics 2004; 1 Pt 1 59-63.
    CrossrefPubMedSearch in Google Scholar
  • 35 Rigby M. Evaluation - the cinderella science of health IMIA Yearbook of Medical Informatics. 2006: 114-120.
    PubMedSearch in Google Scholar
  • 36 Strom BL, Schinnar R, Aberra F, Bilker W, Hennessy S, Leonard CE, Pifer E. Unintended effects of a computerized physician order entry nearly hard-stop alert to prevent a drug interaction: a randomized controlled trial.. Arch Intern Med 2010; 17: 1578-1583. doi: 10.1001/archinternmed.2010.324.
    PubMedSearch in Google Scholar
  • 37 Talmon J, Ammenwerth E, Geven T. The quality of reporting of health informatics evaluation studies: a pilot study.. Stud Health Technol Inform. 2007 ; Pt 1: 193-197.
    PubMedSearch in Google Scholar
  • 38 Talmon J, Ammenwerth A, Brender J, de Keizer N, Nykänen P, Rigby M. STARE-HI - Statement on Reporting of Evaluation Studies in Health Informatics.. Int J Med Inform 2009; 1: 1-9.
    CrossrefPubMedSearch in Google Scholar
  • 39 Tufte E. The Visual Display of Quantitative Information.. Cheshire (Connecticut): Graphics Press; 2001
    Search in Google Scholar
  • 40 van den Brink JL, Moorman PW, de Boer MF, Pruyn JF, Verwoerd CD, van Bemmel JH. Involving the patient: a prospective study on use, appreciation and effectiveness of an information system in head and neck cancer care.. Int J Med Inform 2005; 10: 839-849. doi: 10.1016/j.ijmedinf.2005.03.021.
    CrossrefPubMedSearch in Google Scholar
  • 41 Vandenbroucke JP, von Elm E, Altman DG, Gotzsche PC, Mulrow CD, Pocock SJ, Poole C, Schlesselman JJ, Egger M. Initiative S. Strengthening the Reporting of Observational Studies in Epidemiology (STROBE): explanation and elaboration.. Ann Intern Med 2007; 8: W163-W194.
    PubMedSearch in Google Scholar
  • 42 Vervloet M, van Dijk L, Santen-Reestman J, van Vlijmen B, van Wingerden P, Bouvy ML, de Bakker DH. SMS reminders improve adherence to oral medication in type 2 diabetes patients who are real time electronically monitored.. Int J Med Inform 2012; 9: 594-604. doi: 10.1016/j.ijmedinf.2012.05.005.
    CrossrefPubMedSearch in Google Scholar
  • 43 Winter A, Haux R, Ammenwerth E, Brigl B, Hellrung N, Jahn F. Health Information Systems: Architectures and Strategies.. New York, Berlin, Heidelberg: Springer-Verlag; 2011
    CrossrefSearch in Google Scholar