Earlier this summer, a team of researchers from the Santa Monica, Calif.-based RAND Corporation examined issues around pay for performance initiatives in five different industries: transportation, child care, education, emergency response, and healthcare. The researchers wanted to find out what issues, potential, and pitfalls, the various industries had around performance-based payment of some sort.
A number of interesting findings emerged. Overall, the researchers found limited effectiveness for all kinds of performance-based payment methodologies across all five industries studied. They did find that performance-based payment can under certain circumstances be successful. The six elements that contribute to success, they found, were the following: “a goal that is widely shared among all stakeholders; measures that are unambiguous and easy to observe; incentives that apply to individuals or organizations with control over the relevant inputs and processes; incentives that are meaningful to those being incentivized; few competing interests or requirements; [and] adequate resources to design, implement, modify, and operate” performance-based reward systems, or what researchers called “performance-based accountability systems” (PBASs). When it comes to healthcare, the researchers found overall that pay for performance programs for hospitals and physicians involved incentive payments that were generally too small, relative to overall reimbursements, to spur the kinds of behavior changes that payers and purchasers were hoping for.
Brian Stecher, Ph.D., Acting Director, RAND Education, at the RAND Corporation, led the pay for performance study. He spoke recently with HCI Editor-in-Chief Mark Hagland about the study’s findings.
Healthcare Informatics: What was your methodology for the study?
Brian M. Stecher, Ph.D.: We began with a literature review: We looked at historical work on performance accountability, and in particular, empirical evidence from the five sectors from the past decade: child care, healthcare, education, transportation, and public health emergency preparedness.
HCI: So, how a performance accountability program is designed is key to its effectiveness?
Stecher: One of the main things we found is that you can’t transplant something that works in one context and necessarily have it work in another. So the kind of model developed in child care isn’t necessarily going to work in transportation. And the elements that seem to require the most care include getting the quantitative measures to be fair to the people affected; getting the right incentives in place that motivate and improve performance without corrupting behavior, and that don’t cost more than the improvements that they yield; and getting an agreement about the valued goals that you’re trying to encourage.
HCI: Does healthcare have particularities that make it truly unique?
Stecher: We found that each of the sectors was unique. At the same time, there were enough commonalities there that we were able to do this comparison. Healthcare is not the only industry in which there is joint production going on; that’s also true in education. The situation in education is that while there’s a primary teacher who is ‘first on deck,’ as a primary care physician would be in healthcare, at the same time, the teacher’s performance will be affected by tutors and many others involved in the educational process. And in child care, you have pooled care, and they often work in teams. And maybe in those cases, the right unit of accountability is not the individual worker, but perhaps a group.
HCI: Healthcare is riven with charges of poor data. Is that true in the other industries you studied?
Stecher: Yes; in fact, healthcare and transportation are better off than some of the other industries, in that respect. In healthcare, the issue is cost; if you wanted to do chart review, you could get really granular data. And in transportation, there’s a wealth of data. But there’s much less in child care and in emergency preparedness. So the problems persist everywhere. But in a lot of cases, you take the best data available, which may not be optimal. In child care, you want to know whether a kid is doing better after two years and is prepared for the next level [such as the next level of preschool, or kindergarten]. But in a lot of cases, you have only pupil-to-teacher ratios and staff credentials to go on.