Skip to content Skip to navigation

The Leapfrog CPOE evaluation tool -- "What's the Score?"

August 12, 2008
by anonymous
| Reprints

Work on an online CPOE evaluation tool started 8 years ago when The Leapfrog Group announced its CPOE standard as one of its first three “leaps” in its survey of hospital safe practices. In addition to requiring that physicians enter at least 75 percent of medication orders via a computer system that includes prescriber-error prevention software (CPOE), Leapfrog announced that an evaluation tool would be incorporated into the standard in the future and survey hospitals would be expected to assess their own implemented CPOE.

It’s pretty clear that just implementing any clinical application, such as CPOE, doesn’t mean that clinical decision support is being used. The point of the second part of the Leapfrog standard – namely the evaluation tool - was to provide hospitals with a progress report—as well as a high-level score for public reporting as part of the survey.

Using experts across the country to help develop the content, FCG (now CSC) developed the methodology and the software application. The evaluation tool has been successfully built, tested, piloted, and is now available as part of the 2008 survey process. Although the hospital scores will be not be made public until 2009, the Leapfrog survey results this year will recognize that the hospital has taken the test.

While we were working on this multiyear project – two questions were constantly discussed. Will organizations jump to take the test or follow a “wait and see” approach initially? Also how well have hospitals used the toolsets in CPOE applications to address preventable medication-related adverse events?

The Leapfrog Group held a meeting last week to announce the launch and review very early results. In the short time since the evaluation tool was launched (mid-April), more than 70 hospitals have completed the evaluation. That answers the first question.

Now, how well are CPOE implementations faring? According to my colleague, Jane Metzger, who is part of the team that is analyzing de-identified results, the scores are somewhat predictable and also a bit discouraging: better for basic medication-related decision support like drug-allergy and drug-drug interaction checking than for more advanced screening for problems based on a patient’s age, diagnosis, weight, or renal status.

While these results are somewhat disappointing there is a bright side. We now have a valid tool to determine where the industry’s use of CPOE stacks up against the hopes of the IOM, The Leapfrog Group, and others to improve medication safety and to gauge progress in the future. This is just the beginning, but at least we know where we are!

More information about the evaluation tool is available at http://www.psqh.com/julaug08/cpoe.html

Topics

Comments

Fran,

Thanks for your update. You are right. The potential of CPOE to address the safety goals are being realized in some domains and not in others. There's a real need to understanding and recognizing the progress in detail.

I did want to call your attention to my post last week, containing the reference to the presentation that David Classen gave on this topic in March 2008. My post can be found: HERE


--- here's the excerpt: ---

2. David Classen's 2008 HIMSS presentation entitled, 'Evaluation of Implemented EHRs,' looked at this issue in detail, in the real world. Deployed EMR's were tested using the Leapfrog flight simulator. CDSS scores dropped to the 10% range (50% is passing), as a result of deferred or deliberately disabled CDDS.

----------end of excerpt-----

I tried to raise the distinction between:
1) product strength
2) implementation trade-offs (e.g. turning off alerts), and
3) inherent complexity (e.g. dealing with co-morbid implications for alerts, examples in the original post)

The Leapfrog CPOE evaluation tool is very important and sophisticated work. I'm hopeful that it's predictability will inform and thereby accelerate the industry to safer care delivery, through product improvements that are indicated, better approaches to implementation of products, and enhancements in CDSS to better recognize patient and provider context.

anonymous (not verified)