It's hard to think of a more apt — if gruesome —“what if” example to demonstrate the data integrity challenge. The example comes from Erica Drazen, Sc.D., partner in the Lexington, Mass.-based Emerging Practices division at CSC Corporation (Falls Church, Va.).
“It's what I call the diabetic foot exam example,” she says. In this scenario, a health system wants to analyze the performance of physicians who specialize in caring for diabetic patients, with regard to the routine ordering of a foot exam when a patient comes in for a primary care doctor visit. Logically, the health system would want to track which physicians order foot exams and perhaps set up differential reward programs around such tasks.
But what if Mrs. Jones has already had both of her feet amputated? Her physician's “failure” to order the test should not count negatively, no matter what the data says. Yet few systems that gather process-oriented clinical data are sophisticated enough to take such contingencies into consideration, Drazen points out. And CIOs remain less than confident in the data they're analyzing that comes from multiple sources.
Still, many agree that as pay-for-performance and other quality-related initiatives gain traction, CIOs will need to cement their confidence in the validity and integrity of all types of data in order to survive in the emerging operating environment. Nor are concerns over data integrity abstract and futuristic.
“There was a big brouhaha in Massachusetts recently,” Drazen says, “Because they were going to use claims data for qualifying the physicians; for allowing them to participate in the health plans in that state. Physicians actually filed a lawsuit last fall, claiming the data was so inaccurate” that it should not be used for such purposes, she notes.
Fortunately, more CIOs are becoming aware of the data integrity issue, and those whose organizations are participating in such programs as the CMS/Premier Hospital Quality Incentive Demonstration (HQID) P4P project, are doing something about it. “I think the foot-exam situation is a great example,” says Mike McCurry, vice president and CIO of the 20-hospital Sisters of Mercy Health System, based in St. Louis, “And it exists in many dimensions. As processes change, you get different relevancy of data before and after the process change. And part of what you're trying to do, of course, is to look at data in order to promote wide-scale process change. If the data has changed, it's difficult to appropriately use it.”
McCurry and his colleagues are working on the data integrity problem, not only because of their participation in the HQID project, but as part of their strategic vision of quality.
“The larger scope of what Mercy is doing around quality is really superseding what we're doing with the CMS/Premier program,” he says. “We're on a six-year journey to replace all of our major systems,” and are in the midst of a system-wide implementation of a core clinical IS right now.
With regard to the kinds of issues around production and analysis of meaningful clinical data, “On a technical level, conceptually, it's a fairly straightforward concept, that A must equal A,” says Paul Helmering, executive director of business intelligence at Sisters of Mercy. “But the intuitiveness often is not there. Leaders often dismiss it and move past the problem, only to have to go back to it again later when the data doesn't conform. So it's the single biggest challenge to have to approach with regard to performance improvement work.” Helmering's business intelligence team is currently at work on a major initiative to improve the meaningfulness and usability of data.
Even making sure that an organization accounts for each patient with a single electronic record is a massive challenge that numerous health systems grapple with. “Just getting patients identified correctly, registered correctly, and making sure the electronic records are registered correctly to one patient,” remains a major challenge, says Mitzi Cardenas, vice president and CIO at the 354-bed, two-hospital Truman Medical Centers in Kansas City, Mo. Part of the core problem is simply that, “You still have human beings entering data,” says Cardenas, “And training is challenging.”
But the need to move forward is increasingly important, she says, not only from the pay-for-performance standpoint, “but also because our data is becoming more visible to patients and customers.”
“Quite honestly, there is a lot of bad data” in patient care organizations' databases, says Mark Budd, a partner at CSC. “People pull data into warehouses, where the algorithms aren't correctly written, so it's not clear what level of quality of data you're talking about,” notes the Washington, D.C.-based Budd. “And a lot of data comes in from transactions, where it's fuzzy. So a lot of the focus becomes, who's accountable for making the data accurate?”
Budd, who is managing a project at the federal National Institutes of Health that involves development of a broad-based clinical data repository for analytics, says the path ahead will be long and complicated.
At the prestigious Memorial Sloan-Kettering Cancer Center in Manhattan, vice president and CIO Patricia Skarulis and medical director of information systems and CMIO David Artz, M.D., are working on keeping their data pure.