Economic Studies Part 2: Evaluating the Quality

Introduction

In the first article in this series (EBSJ, volume 3, issue 4), we described some basic terminologies related to health economic studies. In this second and final article, we describe

• available tools for evaluating the quality of economic studies and

• aspects of a high-quality economic study.

As with clinical studies, health economic studies vary in scope and quality. Their usefulness for decision and policy making is linked to their methodological rigor, accuracy, and generalizability. Thus, critical appraisal is important to put the results in the context of study quality.

Economic evaluations identify and compare appropriate alternatives of care, their incremental impact on health outcomes, and their incremental costs. As described in the first part of the series, there are several types of comprehensive medico-economic evaluations.

• Cost-minimization studies consider the cost differences between alternatives of equal effectiveness. This assumes each alternative is equally effective, which is rarely true.

• Cost-benefit studies consider both costs and benefits in monetary terms. Expressing benefits in monetary terms is, however, controversial.

• Cost-effectiveness studies consider differences in costs and differences in effectiveness, but effectiveness is measured variably between studies. For comparisons involving the same disease, condition, or health state, this may be the most meaningful type of evaluation. It is less helpful for comparing studies involving different diseases.

• Cost-utility studies consider differences in costs and outcomes for quality-adjusted survival, most often using the quality-adjusted life year (QALY). Cost-utility studies have the advantage of providing an incremental cost-effectiveness ratio (ICER) expressed as “cost per quality-adjusted life year” (cost per QALY) that eases comparison across multiple studies. This approach incorporates assumptions regarding how each alternative impacts patient's quality of life.

Each employs different methodologies, potentially complicating critical appraisal, but some common criteria can be assessed across studies. No standard, universally accepted method of critical appraisal of economic analyses is currently in use. Across the available methods, there is agreement on several points of methodological quality.[1]

• References

• 1 Hjelmgren J, Berggren F, Andersson F. Health economic guidelines—similarities, differences and some implications. Value Health 2001; 4 (3) 225-250
• 2 Chiou CF, Hay JW, Wallace JF , et al. Development and validation of a grading system for the quality of cost-effectiveness studies. Med Care 2003; 41 (1) 32-44
• 3 Ofman JJ, Sullivan SD, Neumann PJ , et al. Examining the value and quality of health economic analyses: implications of utilizing the QHES. J Manag Care Pharm 2003; 9 (1) 53-61
• 4 Evers S, Goossens M, de Vet H, van Tulder M, Ament A. Criteria list for assessment of methodological quality of economic evaluations: Consensus on Health Economic Criteria. Int J Technol Assess Health Care 2005; 21 (2) 240-245
• 5 Craig D, Nixon J, Armstrong N, Glanville J, Kleijnen J, Drummond M. NHS Economic Evaluation Database Research Team; Response of NHS Economic Evaluation Database Research Team. New BMJ policy on economic evaluations. BMJ 2003; 326 (7386) 445 , author reply 445
• 6 Gerkens S, Crott R, Cleemput I , et al. Comparison of three instruments assessing the quality of economic evaluations: a practical exercise on economic evaluations of the surgical treatment of obesity. Int J Technol Assess Health Care 2008; 24 (3) 318-325
• 7 Gabriel SE, Normand SL. Getting the methods right—the foundation of patient-centered outcomes research. N Engl J Med 2012; 367 (9) 787-790
• 8 Copay AG, Glassman SD, Subach BR, Berven S, Schuler TC, Carreon LY. Minimum clinically important difference in lumbar spine surgery patients: a choice of methods using the Oswestry Disability Index, Medical Outcomes Study questionnaire Short Form 36, and pain scales. Spine J 2008; 8 (6) 968-974
• 9 Lancsar E, Wildman J, Donaldson C, Ryan M, Baker R. Deriving distributional weights for QALYs through discrete choice experiments. J Health Econ 2011; 30 (2) 466-478
• 10 Sach TH, Barton GR, Jenkinson C, Doherty M, Avery AJ, Muir KR. Comparing cost-utility estimates: does the choice of EQ-5D or SF-6D matter? . Med Care 2009; 47 (8) 889-894
• 11 Mehra A, Baker D, Disney S, Pynsent PB. Oswestry Disability Index scoring made easy. Ann R Coll Surg Engl 2008; 90 (6) 497-499
• 12 Andronis L, Barton P, Bryan S. Sensitivity analysis in economic evaluation: an audit of NICE current practice and a review of its use and value in decision-making. Health Technol Assess 2009; 13 (29) iii , ix–xi, 1–61