Are hospital quality metrics causal?
Jason Shafrin
Senior Managing Director, Center for Healthcare Economics & Policy at FTI Consulting; Adjunct Professor, University of Southern California
That is the question asked by a recent NBER working paper by Chandra et al. (2023).
This question is important for a variety of reasons. First, quality measure data collection is expensive. Saraswathula et al. 2023 found that Johns Hopkins Hospital had to report 162 unique quality metrics, and the cost for collecting these data were over $5.6m dollars ($5.0m plus $0.6m in vendor fees. Moreover, understanding whether a metric is causal is important both for value-based purchasing agreements and also evaluation of for whether hospital mergers result in superior quality or patient satisfaction (e.g., see Beaulieu et al. 2020)
A good quality metric should have at least 2 characteristics:
Chandra et al. (2023) aim to test whether hospital quality metrics meet these two criteria.
Methodology
An overview of the Chandra et al. methodology is as follows:
To determine whether hospital quality indicators are causal, we exploit the reassignment of patients as a result of a hospital closure to create quasi-experimental variation in the quality of care that patients receive— if quality indicators are causal, patient outcomes ought to improve when a low-quality hospital closes, and worsen when a high-quality hospital closes (predictive validity). Moreover, the magnitude of the change in patient outcomes should be consistent with the predicted change in hospital quality induced by a closure (forecast unbiasedness).
The authors use 20 years of Medicare claims data for patients hospitalized between 1992 and 2015 for heart-attacks, hip-fractures, pneumonia, congestive heart failure or stroke.? To account for the fact that hospitals may close some units first, before a full facility closure is completed, the authors define closure date as the last year for which a hospital had an admission of a given type. The approach for this population uses a 3-step approach:
领英推荐
The econometric approach largely follows the Chetty et al. (2014) empirical Bayesian methodology, but identification strategy is similar to Doyle et al. (2019) -- which uses ambulance referral patterns to validate the predictive validity for a range of hospital performance measures and Finkelstein et al. (2016)—which uses patient migration to validate geographic variation in health outcomes (mortality) and costs.
Results
Using this approach, the authors find that:
...closures reduced mortality by 0.2 percentage points (off a base of 13%) and reduced readmission by 0.1 percentage points (off a base of 18%), while raising costs and length of stay by about 1%. However, the effect of closure varied widely by zip code depending on the relative quality of the closing hospital. For example, mortality among affected patients in zip codes at the 10th percentile fell by about 1.5 percentage points, while at the 90th percentile they rose by about 1 percentage point
The full paper can be found here.
Further reading:
Originally posted at Healthcare Economist.?
The views expressed herein are those of the author and not necessarily the views of?FTI Consulting, Inc., its management, its subsidiaries, its affiliates, or its other professionals.
Founder | Patient Advocate of the Year 2024 | Health Plan Risk Management | Anti-Bias AI | Helping Plans, Solutions, and Decision-Makers Understand the Complex Patient Journey
11 个月Sharon M. Hibay, DNP, BS, RN thought you'd get a kick out of this.
President, Health Quality Advisors LLC
11 个月Fascinating, and it's great to see a medical economics paper done by real economists. (Sorry, MDs and MPHs folks.) While I appreciated the innovative measurement, I had a few questions. 1) Readmissions to where? When a hospital does something wrong, the patient is not necessarily readmitted to that hospital; 2) closing vs. non-closing is a choice that provides clarity; however, as we all know, it takes a lot to close a hospital. Indeed, poorly performing hospitals may be in areas where the economic interest in keeping them open outweighs reputation. Moreover, as a number of academic articles have noted, non-fatal medical errors can be profitable (such as in surgery), helping keep the hospital open!; 3) it would be interesting to separate out all-cause patient safety measures, such as Leapfrog. There's no doubt we have a surplus of quality measures; 4) Hey, economists, wouldn't it be nice to at least assign some sort of economic value to lives improved or saved? Although I guess if it's Medicare patients, you could say, "Great -- we're saving Social Security." But what about the economic value to a family of not knocking off Grandma! Anyway, I always like to see some human element thrown in. Thanks! Michael
Global Strategist in Health, Wealth and Technology
11 个月Very interesting study. Thanks for sharing.