Assessment of Data Quality of and Staff Satisfaction with an Electronic Health Record System in a Developing Country (Uganda)

Summary

Objectives: To assess if electronic health record systems in developing countries can improve on timeliness, availability and accuracy of routine health reports and staff satisfaction after introducing the electronic system, compared to the paper-based alternative.

Methods: The research was conducted with hospital staff of Tororo District Hospital in Uganda. A comparative intervention study with qualitative and quantitative methods was used to compare the paper-based (pre-test) to the electronic system (post-test) focusing on accuracy, availability and timeliness of monthly routine reports about mothers visiting the hospital; and staff satisfaction with the electronic system as outcome measures.

Results: Timeliness: pre-test 13 of 19 months delivered to the district timely, delivery dates for six months could not be established; post-test 100%. Availability: pretest 79% of reports were present at the district health office; post-test 100%. Accuracy: pre-test 73.2% of selected reports could be independently confirmed as correct; post-test 71.2%. Difficulties were encountered in finding enough mothers through direct follow up to inquire on accuracy of information recorded about them. Staff interviews showed that the electronic system is appreciated by the majority of the hospital staff. Remaining obstacles include staff workload, power shortages, network breakdowns and parallel data entry (paper-based and electronic).

Conclusion: While timeliness and availability improved, improvement of accuracy could not be established. Better approaches to ascertaining accuracy have to be devised, e.g. evaluation of intended use. For success, organizational, managerial and social challenges must be addressed beyond technical aspects.

Keywords

Evaluation - computerized medical record systems - health records quality - staff satisfaction - qualitative and quantitative comparative study

• References

• 1 Lippeveld T, Sauerborn R, Bodart C. (eds). Design and implementation of health information systems. Geneva:: World Health Organization;; 2000. ISBN 92-4-156199-8.
• 2 Braa J, Monteiro E, Sahay S. Networks of Action: Sustainable Health Information Systems across developing countries. MIS Quarterly 2004; 28 (03) 337-362.
• 3 World Bank.. World Development Report 1993: Investing in Health. Oxford University Press; 1993. ISBN 0-19-520889-7, ISBN 0-19- 520890-0, ISSN 0163-5085.
• 4 WHO, Regional Office for Africa, Brazaville (WHO/AFRO).. Report on Strengthening National Health Information Systems: Workshop on the use of Geographical Information System HealthMap. Harare, Zimbabwe: July 1-4, 2002. WHO; 2002
• 5 WHO, Regional Office for Africa, Brazaville.. Priority Interventions for Strengthening National Health Information Systems. WHO 2004
• 6 Kintu P, Nanyunja M, Nzabanita A, Magoola R. Axios Group, World Health Office for Uganda,. et al. Development of HMIS in poor countries: Uganda as a case study. UMU Press 2005; 3 (01) 46-53.
• 7 Ministry of Health, Department of Planning & Development, Monitoring & Evaluation Unit.. Assessment of the Health Information System in Zambia. Republic of Zambia, Ministry of Health; 2004
• 8 Nsekuye P. Community Health Department, St.. Francis Mutolere Hospital, Kisoro, Uganda. Data for the Boss: Evidence of non-use of health management information systems (HMIS) data in Bufumbira east health sub-district, Kisoro district. UMU Press 2007; 2 (01) 3-12.
• 9 Ministry of Health, 1985.. The Health Information System for Uganda: a guide for health workers. as cited in Kintu P, Nanyunja M, Nzabanita A, Magoola R. Axios Group, World Health Office for Uganda, et al.. Development of HMIS in poor countries: Uganda as a case study. UMU Press 2005; 3 (01) 46-53.
• 10 Ministry of Health, 1997.. Review of the Health Management Information System for Uganda: final report. as cited in Kintu P, Nanyunja M, Nzabanita A, Magoola R. Axios Group, World Health Office for Uganda,. et al. Development of HMIS in poor countries: Uganda as a case study. UMU Press 2005; 3 (01) 46-53.
• 11 Resource Center, Ministry of Health, 2001.. Indicator Manual for Monitoring HSSP Performance. as cited in Kintu P, Nanyunja M, Nzabanita A, Magoola R. Axios Group, World Health Office for Uganda,. et al. Development of HMIS in poor countries: Uganda as a case study. UMU Press 2005; 3 (01) 46-53.
• 12 Dick RS, Steen EB, Detmer DE. editors. Committee on Improving the Patient Record, Institute of Medicine. The Computer-Based Patient Record: An Essential Technology for Health Care, Revised Edition. National Academy of Sciences; 1997 ISBN 0-309-05532-6.
• 13 Ball MJ, Garets DE, Handler TJ. Leveraging Information Technology Towards Enhancing Patient Care and Culture of Safety in the U.S. Methods Inf Med 2003; 42 (05) 503-508.
• 14 Institute of Medicine.. To Err Is Human: Building a Safer Health System. National Academy of Sciences; 2000
• 15 Mbananga N, Madale R, Becker P. Evaluation of Hospital Information System in the Northern Province of South Africa – Using Outcome Measures. Health Systems Trust 2002 (May).
• 16 Nohr C. Virtual Center for Health Informatics, Department of Development and Planning, Aalborg University, Denmark. Evaluation of Electronic Health Record Systems. Methods Inf Med 2006; 45 (01) 107-113.
• 17 Jaspers MWM, Knaup P, Schmidt D. The Computerized Patient Record: Where Do We Stand? IMIA Yearbook of Medical Informatics 2006. Methods Inf Med 2006; 45 (01) S29-39.
• 18 Ellingsen G, Monteiro E. Big Is Beautiful: Electronic Patient Records in Large Norwegian Hospitals 1980s-2001. Methods Inf Med 2003; 42: 366-370.
• 19 Jones MR. “Computers can lead people on Mars, why can’t they get them to work in a hospital?” Implementation of an Electronic Patient Record System in a UK Hospital. Methods Inf Med 2003; 42: 410-415.
• 20 Peterson HE. From Punched Cards to Computerized Patient Records: A Personal Journey. Methods Inf Med 2006; 45 Suppl S180-186.
• 21 Galdwin J, Dixon RA, Wilson TD. Implementing a new health management information system in Uganda. Health Policy and Planning 2003; 18 (02) 214-224.
• 22 Chaulagai CN, Moyo CM, Koot J, Moyo HBM, Sambakunsi T, Khunga FM. et al. Design and Implementation of a Health Management Information System in Malawi: issues, innovations and results. Health Policy and Planning 2005; 20 (06) 375-384 doi:10.1093/heapol/czi044.
• 23 Cibulskis RE, Hiawalyer G. Information Systems for Health Sector Monitoring in New Papua Guinea. Bulletin of the World Health Organization 2002; 80: 752-758.
• 24 The Republic of Uganda.. The Health Management Information System. Health Unit Procedures Manual. Volume I. Ministry of Health, Uganda. June 2005
• 25 Uganda Population Census of 2002.. Available from: http://www.statoids.com/uug.html (cited 2007 October 5).
• 26 Granneman S. Linux vs. Windows Viruses. Security Focus, 2003. Available from: http://www.securityfocus.com/columnists/188 (updated 2003 October 2; cited 2007 October 5).
• 27 Gomez D, Höpfner H. Postgres and MySQL in direct comparison. Head to Head. Linux Magazine. August 2001. Available from: http://www.linux-magazine.com/issue/08/023-PostSQL.pdf (cited 2007 October 5).
• 28 Valdes R, Drakos N, Driver M. IBM/Zend Initiative Offers Open-Source Web Development Option. Gartner Research. March 3, 2005. ID Number: G00126609.
• 29 Vasudevan A. PHP HOW-TO; Section 5.1 “Advantages of PHP over Java”. v27.6, 05 Oct 2002 Available from: http://www.faqs.org/docs/Linux-HOWTO/PHP-HOWTO.html
• 30 Stillger R, Misselwitz B. Messung von Qualität: Probleme und Lösungsansätze. Gynäkologe 2001; 34: 89-92.
• 31 Hammann P, Erichson B. Marktforschung, 3. Aufl. Stuttgart Jena, New York: UTB; 1994. pp 105ff.
• 32 Resource Center, Ministry of Health, 2002.. Statistical Abstract (Working paper). Ministry of Health, Uganda. April 2003
• 33 Busharizi P. Drought, “Loadshedding to cost economy billions”. The New Vision article. Wednesday, February 15, 2006. Available from: http://www.newvision.co.ug/D/8/220/482098 (updated 2006 February 15; cited 2007 October 8)
• 34 Austin J. “Loadshedding a big lesson to us”. The New Vision article. Thursday, March 16, 2006. Available from: http://www.newvision.co.ug/D/8/459/487473 (updated 2006 March 16; cited 2007 October 8).
• 35 Stausberg J, Koch D, Ingenerf J, Betzler M. Comparing Paper-based with Electronic Patient Records: Lessons Learned during a Study on Diagnosis and Procedure Codes. J Am Med Inform Assoc 2003; 10: 470-477. DOI 10/1197 jamia.M1290.
• 36 Williamson L, Stoops N, Heywood A. Developing a District Health Information System in South Africa: A Social Process or Technical Solution?. Medinfo 2001; 10: 773-777.
• 37 Brender J, Ammenwerth E, Nykänen P, Talmon J. Factors Influencing Success and Failure of Health Informatics Systems – a pilot Delphi Study. Methods Inf Med 2006; 45: 125-136.
• 38 Mihalas GI, Bazavan M, Farcas DD. Implementation of Health Information Systems in Romania. Methods Inf Med 2006; 45: 121-124.
• 39 van der Meijden MJ, Solen I, Hasman A, Troost J, Tange HJ. Two Patient Care Information Systems in the Same Hospital: Beyond Technical Aspects. Methods Inf Med 2003; 42: 423-427.
• 40 Faber MG. Design and Introduction of an Electronic Patient Record: How to Involve Users?. Methods Inf Med 2003; 42: 371-375.
• 41 Berg M, Aarts J, van der Lei J. ICT in Health Care: Sociotechnical Approaches. Methods Inf Med 2003; 42: 297-301.
• 42 Heeks R, Mundy D, Salazar A. Why Health Care Information Systems Succeed or Fail. Institute for Development Policy and Management, University of Manchester; June 1999. ISBN 1-90251825X.
• 43 Declaration of Innsbruck – Results from the European Science Foundation Sponsored Workshop on Systematic Evaluation of Health Information Systems (HIS-EVAL). April 4-6, 2003. In: Ammenwerth E, Brender J, Nykanen P, Prokosch HU, Rigby M, Talmon J. et al. (eds). Visions and strategies to improve evaluation of health information systems; Reflections and lessons based on the HIS-EVAL workshop in Innsbruck. Int J Med Inf 2004; 73 (06) 479-491.
• 44 Rigby M. Evaluation – The Cinderella Science of ICT in Health. Methods Inf Med 2006; 45 (01) 114-120.
• 45 Ammenwerth E. Health Technology Assessment. Findings from the Section on Assessing Information Technologies for Health. Methods Inf Med 2006; 45: 16-19.
• 46 Ammenwerth E, Shaw NT. Bad Health Informatics Can Kill – Is Evaluation the Answer?. Methods Inf Med 2005; 44: 1-3.
• 47 Description of Health Units by Ministry of Health, Uganda. Available from: http://www.health.go.ug/health_units.htm (cited 2007 October 8).
• 48 The Republic of Uganda.. 1996. The Health Management Information System Manual, Volume I: The Health Unit Manual. Ministry of Health, Uganda. July 1996
• 49 Maine D, Akalin MZ, Ward VM, Kamara A. The Design and Evaluation of Maternal Mortality Programs, Centre for Population and Family Health, School of Public Health,. Columbia University,; 1997
• 50 Freedman LP, Waldman RJ, De Pinho H, Wirth ME, Chowdhurry AMR, Rosenfield A. Who’s got the power? Transforming health systems for women and children. London:: Earthscan Publications;; 2005. p 185.
• 51 Wetter T. To decay is system: The challenges of keeping a health information system alive. Int J Med Informatics 2007; 76: 252-260.