Skip to content Skip to navigation

Looking at Outcomes Measurement, HSC Researchers Ask: How Many Measures Are Too Many?

May 12, 2013
by Mark Hagland
| Reprints
The Center for Studying Health System Change releases a policy analysis on outcomes measurement that holds many implications for healthcare CIOs and CMIOs

On April 25, the Washington, D.C.-based center for Studying Health System Change (HSC) released a new policy analysis, entitled “Hospital Quality Reporting: Separating the Signal from the Noise,” written by Emily R. Carrier, M.D., an HSC senior researcher, and Dori A. Cross, an HSC health research assistant. As HSC noted in a press release announcing the release of that policy analysis, “Amid the proliferation of quality measures, reporting requirements and transparency efforts, purchasers often find it difficult to separate the signal from the noise when determining what hospital quality measures are important, how to interpret and use quality information in a meaningful way, and how to present useful and actionable information to consumers.”

And, as authors Carrier and Cross note in the paper, “With U.S. healthcare costs high and rising, purchasers increasingly are seeking to identify high-value hospitals that deliver good care at a reasonable price. Some payers,” the authors note, “are incorporating clinical quality measures into health plan contracting, and benefit designs to alter provider networks and patient cost-sharing to drive patients toward higher-performing hospitals.” Yet a tremendous set of challenges exists in attempting to streamline provider reporting requirements and support consistency in definitions and practices, in the outcomes measurement arena.

There are many extremely useful elements in the paper, among them the first table, “Types of Hospital Quality Measures,” which summarizes all the different types of quality measures being used these days, with remarks on the “pros” and “cons” of each type of measure.

Shortly after the release of the policy analysis, Dr. Carrier spoke with HCI Editor-in-Chief Mark Hagland regarding the findings in the policy paper, and the implications of those findings for healthcare CIOs, CMIOs, and other IT leaders. Below are excerpts from that interview.

What was your overall goal in doing this analysis?

Our overall goal was to get a big-picture view of quality measurement for purchasers who might not really have been able to look at the issue in depth before, but who might be interested in picking it up.

Emily Carrier, M.D.

Were you surprised by anything, as you began to study all of this in depth?

Not really; I come at it from a clinical background, so experiencing this on a practice level, you sort of experience these coming in as different initiatives, coming one after the other. A performance measure becomes something important to your organization, and everyone’s thinking about it and working on it, and it becomes encrypted into your practice, and then another one comes along, and it doesn’t feel as though there’s a centralized approach to anything. And that reflects our health care system’s approach of “let a thousand flowers bloom.” There’s a national quality strategy that’s been developed by the national government, but it hasn’t yet been reflected in how these measurement efforts are implemented on the practice or organizational level.

Are clinicians faced with having to be graded on simply too many outcomes measures at once? That has been one of the biggest complaints from physicians for years now.

When you say too many, I ask, too many compared to what? Certainly there are a lot; and certainly there are subtle differences that would lead a provider to think that there are too many. Are we truly capturing the important dimensions of quality? That’s not clear. And there may be too many functionally, and yet we may still not be capturing the most important dimensions of quality. So rather than saying there are too many or too few, maybe we’re just not capturing the right dimensions. There’s a classic story about quality: you’re walking down the street and there’s a drunken man crawling around a lamppost, and you say, can I help you? And he says, I’m looking for my teeth, I’ve lost them. And you ask, are your teeth somewhere here around this lamppost? And he says, no, they’re further away, but he’s looking near the lamppost because that’s where the light is. And if what you have is medical claims, you’ll look at what you can capture in claims; if you have chart audits, that’s where you look. And now you have EHRs [electronic health records] to work with, and there’s great potential there, but there are questions also.

There is great potential in EHRs, but also, there’s a documentation burden on physicians, in terms of having to document so many things in the medical record; and there are the meaningful use requirements related to quality as well. Physicians are feeling particularly burdened these days, because of the combination of the meaningful use and value-based purchasing mandates.

Well, there are different ways of getting things done. And we may say that clinicians have to do the heavy lift; if we need a piece of information, we’re going to require that data be entered in a standardized way. Another way might be to say, we’re going to have the EHR developers do the lift, and work towards trying to create a 360-degree view beyond what can be captured through a single provider’s EHR, taking better advantage of what clinicians include in things they write down in freetext, things that are recorded automatically, etc., capture other aspects of care. And then there’s the full spectrum in between. To really fully realize the potential of EHRs, a lot of effort will be required, and it remains open as to on whom the main burdens will fall.