Hospital benchmarking based on administrative data: methods, experiences, reactions

Table of Contents

Key words

hospital report cards - administrative data - patient safety indicators - quality indicators

Schlüsselwörter

Krankenhaus-Zeugnis - Routinedaten - Patientensicherheits-Indikatoren - Qualitätsindikatoren

The Fraser Institute’s Hospital Report Cards provide information on the performance of all of the acute care hospitals in the Canadian provinces of Ontario (except for the Hospital for Sick Children in Toronto) and British Columbia. Reports for other provinces are planned for the coming years. All of the information in the report cards can be accessed in a convenient and interactive way through our websites, www.fraserinstitute.org and www.hospitalreportcards.org.

We set out to create a report card that is easy to understand and accessible by the public, where individuals are able to look up a given condition or procedure and compare death rates, volumes of procedures, rates of adverse events, and utilization rates for their hospital to those of other hospitals. The basis of our methodology is the US Agency for Healthcare Research and Quality’s (AHRQ) Patient Safety Indicators and Inpatient Quality Indicators, from which we selected 39 indicators that can be reliably calculated using Canadian administrative data. Risk-adjustment is handled by the AHRQ software using the 3M™ APR DRGs. Indicators apply only to acute-care conditions and procedures for inpatient care, and are calculated for both acute-care hospitals as well as municipalities (based on patient residence postal codes).

Our primary source of anonymous patient-level administrative data is the Canadian Institute for Health Information’s (CIHI) Discharge Abstract Database. These data are widely used to produce various CIHI reports, and form the basis for many journal articles. The Fraser Institute spent two years developing the methods, databases, and computer programs required to adapt the AHRQ measures to Canadian circumstances. This work has been both internally and externally peer-reviewed.

In addition to separately reporting the 39 indicators, the Hospital Report Card also includes an aggregate measure of hospital performance called the Hospital Mortality Index. This index consists of nine indicators (equally weighted), and examines overall performance of a hospital or municipality across indicators that measure death rates.

The anonymity of hospitals in the report cards remains a key concern for researchers and the public. The administrative records held by CIHI are only available to the general public without individual institution identifiers, unless those institutions have granted express permission to be identified. Perhaps not surprisingly, many hospitals have chosen to remain anonymous.

Of Ontario’s 136 acute-care hospitals, 30 – representing 4.9 % of inpatient records in Ontario – granted us authorization to identify them by name in the 2008 report card. For the first report card in Ontario (published 2006), we were authorized to identify 43 hospitals, representing 41 % of inpatient records in the latest year of data reported. None of British Columbia’s 95 acute-care hospitals granted us authorization to identify them by name in the first report there (published 2008).

The anonymity of hospitals is a setback to accountability, the empowerment of patients, and the ongoing commitment of hospitals to quality improvement and transparency. This refusal by hospital administrators to allow their performances to be measured and reported on is unacceptable. On the other hand, the Ontario hospitals who voluntarily agreed to be identified should be commended for their efforts to empower patients with information regarding the health care they receive and for their ongoing commitment to quality improvement through accountability and transparency.

The popular media generally seem to agree with this perspective. Media attention surrounding the release of the report in British Columbia in particular was extensive, with many reporters, talk radio hosts, and media commentators/interviewed experts questioning the decision to withhold the names of hospitals from the public. Importantly, the services being examined in the Hospital Report Card are delivered in public hospitals through Canada’s tax-funded monopoly health insurance program.

The Hospital Report Card is a valuable product for the public and for providers of health care. The report increases accountability in the government-managed, publicly funded health care sector and has the potential to improve quality of care. In order for these benefits to accrue to Canadians however, it is critical for governments and hospitals to accept that their performances should be both measured and publicly reported.

Author’s declaration: The author does not have any financial interests or conflicts of interest relevant to the subject of this manuscript.