CCHIT & the DMV | [node:field-byline] | Healthcare Blogs Skip to content Skip to navigation


August 14, 2009
by aguerra
| Reprints

Certification is moving in the right direction, but perils lay around every corner.

“Just take it down the street to Jerry,” said my father, regarding where I should get the 1976 Oldsmobile Delta 88 I had inherited from my grandmother inspected.

Sure, taking the car to Jerry’s repair station had its disadvantages. For one, he charged $75 a sticker, he lacked certain social graces (like acknowledging your presence), and you could wait three hours for the smallest repair because he was easily distracted (‘Oh God, please don’t let the phone ring again’). However, the allure of going to Jerry’s station to buy an inspection sticker was just that — you bought a sticker. Going to the DMV for a free test/sticker was fine for those with new cars, but my 20-year-old classic needed special evaluation from an “authorized inspection location” like Jerry’s.

Over the years, the DMV has caught on to the inspection techniques favored by Jerry and his cohorts, implementing more stringent “tamper-proof” methods. Of course, just how their emissions-testing machine knows which vehicle it’s checking is beyond me. The bottom line is that stations can make $75 a pop for inspecting cars, and nobody tested there is interested in a genuine evaluation of their vehicle.

Inspection and certification programs are tricky to design. When it comes to CCHIT (possibly soon-to-be HHS) Certification, the most recent proposal from the HIT Policy Committee Workgroup on Certification & Adoption reveals some guiding principles.

CCHIT has been the sole body which both establishes the criteria for and tests EMR software. However, the federal workgroup recommends those functions be formally separated, with HHS/CMS/ONC establishing the criteria, and groups like CCHIT competing to perform the testing (with input on how that should be handled likely coming from NIST).

This is a solid first step, as CCHIT was far too nebulous and conflicted (at least in appearance) to hold the industry’s confidence as the one government-mandated body controlling the massive EMR certification pipeline. Injecting competition will lower price and increase quality.

But the devil, as they say, will be in the details. Say the Workgroup’s recommendations are adopted. I see no problem with HHS absorbing the criteria-creation and maintenance functions, with CCHIT offering HHS Certification testing. But who might be bold enough to go up against CCHIT, and why would a vendor bring their EMR testing business elsewhere?

First off: price. A new entrant might have less infrastructure — real estate and payroll — or be willing to take less profit. Additionally, newcomers might be simply more efficient, offering faster results, enhanced ancillary services, or greater convenience.

In my interview with Certification & Adoption Workgroup Member Paul Egerman, he made it clear HHS Certification testing would carry the same weight regardless of where it is performed. For this to be valid, HHS will need to do a superior job of testing (and retesting) the testers.

You see, like Jerry, the testers will be tempted to choose revenue over rejection. How many people have seen trucks bellowing smoke on the roads while bearing new inspection stickers? Where do you think those gold stars came from? If HHS doesn't ensure its seals are above reproach, 'Where'd you get your sticker,' could enter into product discussions.

The workgroup handling certification is on the right path, but pitfalls linger. Let’s make sure certification testing is widely available and reasonably priced. But let’s also make sure the HHS seal isn’t devalued because of guys like Jerry.



Readers should remember that CCHIT and other certifications are "on-the-shelf" tests. That would be fine if "as-implemented" was the same as "as-demonstrated and contracted." As shown by our friends at CSC (formerly FCG), there is wide variation in the ability of CPOE systems to detect and prevent harm and/or death than transcends certification.

I've covered this before here.   The slide above is from the 2008 update.

Over 75 CPOE implementations including large volumes of the commercially available systems were tested.  He mentions that Epic, Cerner, and Eclipsys were well represented in those numbers, but that no vendor was immune to a poor implemenation of a good product. 

None of them, as implemented, caught 100% of harmful errors.  This is despite the fact that all of them were designed to.  The challenge, as Classen points out, is that implementation done well is a multi-year, iterative process.  And, despite processes like regression testing, adding new functionality, modules, and content, as well as taking new software versions can and does break things that previously worked.

In August of 2008, long before we dreamed of ARRA/HITECH stimuli, Classen warned that the increasingly popular, rapid implementation methodologies were particularly vulnerable to defects that fail to address critical safety test cases.

I was convinced before that.  Certification without Evaluation will not help hospitals, clinics and other care delivery centers become extremely safe, highly reliable places to receive care.

thanks for your comment Joe. Classen's warnings about short implementation timelines make HITECH's requirements for 2011 even more troubling. Many will put in CPOE, but will it be done "right" and will the benefits be achieved. For example, right now meaningful use for 2011 (hospitals) requires entering at least 10 percent of all order electronically, but that doesn't ensure those orders are entered correctly, that doesn't ensure that the clinician entering the order was presented with the proper alert in a actionable format, that doesn't mean a medication error was averted. It's going to be very interesting to see how this plays out.