Measuring and reporting mortality in hospital patients aims to develop national indicators of in-hospital mortality and is one of several projects conducted for the National Indicators Project commissioned by the Australian Commission on Safety and Quality in Health Care. The project has two parts: a literature review focusing on methods for analysing and reporting in-hospital mortality, and a modelling project aimed at establishing national indicators of in-hospital mortality that can be implemented now, and in the future.
Papers on in-hospital mortality have been appearing in the scholarly literature since the middle of the 19th century. A large and growing body of modern literature describes the methods used to measure in-hospital mortality to allow comparison of mortality levels between different hospitals. Valid comparisons require methods that adjust for the differing risks of patient mortality that arise from hospitals having patients with different mixes of illnesses.
There is an emerging international consensus on which measure to use (the risk-adjusted Hospital Standardised Mortality Ratio, HSMR), on patient characteristics (such as age and diagnosis) to be included in risk-adjustment models, on modelling methods, and on types of cases to exclude (e.g. palliative care cases). Routinely collected data from good quality systems appear to provide an adequate basis for measuring in-hospital mortality, though discussion continues about data quality. Risk-adjusted in-hospital mortality rates–calculated using routinely collected data–are now reported regularly and publicly in several countries or jurisdictions within countries (United Kingdom, The Netherlands, Canada, and Queensland, Australia).
Three main methods are used for presenting comparative in-hospital mortality data: tables, caterpillar plots and funnel plots. For individual hospitals, the methods generally feature the ratio between the actual or observed mortality rates and the expected rates calculated from the models. Because there is some random variation in mortality rates, and expected rates fall within a range, the confidence intervals for the expected rates are usually also presented.
Longitudinal analysis of in-hospital mortality is an emerging and powerful new theme in the literature.
Measuring in-hospital mortality in Australia
The routinely collected data from the Australian National Hospital Morbidity Database were analysed. We applied a method used in Canada, England and the Netherlands, and referred to in this report as the risk-adjusted Canadian referred mortality (RACM) model. Logistic regression modelling of in-hospital mortality was used to calculate expected mortality: adjusting risk according to principal diagnosis, age, sex, comorbidity, length of stay, emergency or elective admission status and whether transferred from another hospital. The expected mortality estimate for each hospital was then combined with observed deaths to calculate risk-adjusted HSMRs.
The model was tested to determine how well it predicted or explained the actual variation in mortality rates.
HSMR analysis was conducted on three groups of cases, which exemplify types of general-purpose indicators of in–hospital mortality:
- high-risk cases (20% of cases, 80% of in-hospital deaths)
- lower risk cases (all other in-scope cases; that is, the other 80% of cases including 20% of in-hospital deaths)
- all cases and all in-hospital deaths.
Data for 1 year were analysed initially. Longitudinal analysis was then done using 3 years of data. This was a two step process. The first step was to calculate risk-adjusted HSMRs in a similar way to the 1–year analysis. The second step was two-stage multi-level logistic regression.
The hospital peer group classification developed by the AIHW was used to group hospitals for comparisons.
Overall, the results demonstrated that, using the Australian data, the RACM model predicted or explained the variation in mortality rates to a similar extent as models reported in the international literature. Some differences in the strength of the model were apparent when applied to the three mortality groups (80%, 20% and 100%): with better prediction of mortality rates for the 20% and 100% groups.
Single-year analysis (2005-06)
The single-year analysis resulted in the production of HSMRs and confidence intervals for public hospitals in peer groups. They are presented using HSMR ranked tables, funnel plots and caterpillar plots. Funnel plots illustrated that some hospitals had HSMRs that were relatively high or low compared with peer hospitals.
Longitudinal analysis (2004-05 to 2006-07)
The longitudinal analysis showed that most variation in HSMRs was between different hospitals, with much less variation between repeated measurements for the same hospital. The lack, on the whole, of large variation between measures of HSMR for the same hospital suggests that values largely reflect the phenomenon of interest (mortality rates), and are not dominated by 'noise' in the data. This is less true for peer groups of small hospitals.
The results presented for the longitudinal analysis demonstrate a modest decline in overall risk-adjusted mortality during the 3–year period. This is similar to the findings of a recent Dutch study using the same method. Although replication of analysis and refinement of the method used should be undertaken before too much weight is place on this finding, the possibility remains that it is a true decline. If so, perhaps an increased emphasis on hospital safety in recent years is beginning to have a demonstrable effect on in-hospital mortality.
This project shows that indicators of in-hospital mortality can now be produced using the Australian National Hospital Morbidity Database. The present study produced indicators based on the three mortality groups specified above, reported by hospital in public hospital peer groups. Our findings suggest that the available data are generally sufficient for this purpose.
How should the HSMRs be used? Variations in hospital mortality should be viewed as screening tests rather than being diagnostic of poor safety or quality. High or rising HSMRs signal that a problem might exist and that further investigation is required. Low or falling HSMRs might signal good performance, from which lessons could be learned.
The report also describes a refinement to the RACM model–referred to as the elaborated risk-adjusted mortality (ERM) model–which consistently predicted mortality rates better than the RACM model. This model could be further investigated for its potential to generate indicators of in-hospital mortality.
Further work is also warranted on several matters where data limitations prevented us from undertaking desirable aspects of analysis.
Data matching to include deaths up to 30 days after discharge is technically feasible in Australia, as is internal linkage of the data on multiple episodes of care for individual patients. Both of these forms of linkage are routine parts of data linkage activities in some states. They could be used for a project to test these refinements of the data underlying HSMR, in preparation for later use of linked data for national indicators.
The lack of a hospital identifier for many private hospitals prevented analysis of this sector– a limitation that needs to be overcome in future analyses.
Emerging data developments (national coding of conditions 'present on admission' to distinguish pre-existing comorbidities from complications of care) and analytical innovations (e.g. use of Bayesian regression, especially for data from small hospitals) are also likely to improve results.