When Lucile Packard Children's Hospital at Stanford (LPCH), in Palo Alto, Calif., went live with computerized physician order entry (CPOE) in the fall of 2007, healthcare IT leaders there were able to “stand on the shoulders” of other children's hospitals that had rolled out CPOE in the years immediately prior. Indeed, most of those that had already implemented CPOE were, like LPCH, customers of the Kansas City-based Cerner Corp.: Children's Hospital of Pittsburgh of UPMC (live in 2002); Seattle Children's Hospital and Children's Hospital of Los Angeles (2003); Children's Hospital Orange County (2005); and Children's Hospital Boston (2007). So there was already experience with some of the particular challenges of CPOE go-lives in the pediatric environment, including with a specific vendor solution, says Christopher A. Longhurst, M.D., LPCH's CMIO.

But there were also some concerns, given that James Levin, M.D., Ph.D., and his colleagues had seen temporarily increased mortality levels at Children's Hospital of Pittsburgh, following that organization's CPOE go-live several years earlier, which was described in a December 2005 article in Pediatrics. Levin and his colleagues worked assiduously to determine the causes of the mortality level increases, and were able to reverse those effects at their organization.

In addition, the experiences of all those children's hospitals that had implemented Cerner's core CPOE system strongly informed Longhurst and his colleagues at LPCH. As a result of all the preparations involved, LPCH did not experience even a temporary increase in mortality rates; instead, over time, it experienced a sustained decrease in mortality rates, as Longhurst and eight co-authors have described in a July 2010 article in Pediatrics, entitled, “Decrease in Hospital-wide Mortality Rate After Implementation of a Commercially Sold Computerized Physician Order Entry System.” Indeed, adjusting for a number of complicating factors, Longhurst and his colleagues determined that the hospital had sustained a 20-percent decrease in mortality over the 18-month period of the study that led to the 2010 Pediatrics article.

Further, Levin and Longhurst co-presented at the annual AMDIS (Association of Medical Directors of Information Systems) conference, held in July last year in Ojai, Calif. Their presentation, “Evidence-based EMR Implementation: Achieving Meaningful Outcomes and Avoiding Unintended Consequences,” was widely regarded as the most impactful of the conference by the assembled CMIOs.

Longhurst spoke recently with HCI Editor-in-Chief Mark Hagland regarding what he and his colleagues at LPCH-and elsewhere-have learned about how to effectively implement CPOE in a pediatric hospital setting. Below are excerpts from that interview.

Healthcare Informatics: Did you and your colleagues create extensive customizations from the core Cerner CPOE solution when you rolled out?

Christopher A. Longhurst, M.D.: Yes, a tremendous amount.

HCI: And you were able to learn from the implementations at those other children's hospitals?

Longhurst: Yes, particularly from the experiences of the Seattle, L.A. and Boston children's hospitals; and we feel a tremendous debt to all of them.

HCI: Were there any special or particular challenges in terms of the go-live itself?

Longhurst: One of the important things-we have some of the highest acuity of any children's hospital in the United States. In fact, in the fall of 2009 through the fall of 2010, we ranked as having the highest-level acuity in case mix of any children's hospital in the U.S. We're really a quaternary children's hospital.

HCI: What were your overall goals in leveraging CPOE for care improvement?

Longhurst: Pre-go-live, our project had really been pushed to the board as a quality and safety effort; and I credit my colleague Paul Sharek, M.D., our chief quality officer, in helping to promote that. So this had always been looked at as a quality and safety project, and really, from the outset of the capital project, it was viewed as clinical transformation leveraging information technology to produce process redesign.

HCI: So your study was a natural outcome of that?

Longhurst: Well, prior to the study, what we had put together was a dashboard for our board of directors with 32 different indicators, specific to the project. The 32 different indicators included a variety of things; some were metrics that we had already been tracking, and that we then included on the CPOE/EMR dashboard; others were specific. So for example, we tracked CPOE adoption rates and verbal order rates very closely. We also tracked turnaround times for laboratory, medication and radiology orders. In some cases, the metrics were unique to the project; in other cases, such as with lab turnaround times, we had been tracking time from sample accession to result availability; that's sort of a classic lab measure. But we wanted to look at the time from order to result for a stat, one-time order.

And then finally, another piece of that dashboard was actually our mortality data. And that was on our EMR dashboard because of the Pittsburgh Children's experience. Our project had kicked off in January 2006, which was a month after the publication of the Children's Pittsburgh article about their experience of having a bump in mortality following their go-live in the 2002-2003 period, though published in December 2005.

HCI: So you were concerned about that?

Longhurst: Yes, we were particularly concerned about the impact on children's hospitals, and particularly among children's hospitals that used Cerner.

HCI: How did your study come about-the one that led to the 2010 article that you and your colleagues authored, in Pediatrics?

Longhurst: It really was an outgrowth of the board of directors dashboard work. The mortality bump from Children's Pittsburgh really was all from the first three months following go-live. And Mark Del Beccaro [Mark Del Beccaro, M.D., CMIO, Seattle Children's Hospital] produced data showing that there hadn't been a bump in mortality. So we really stood on the shoulders of those who came before us, in particular Pittsburgh, Seattle, and Los Angeles, that they had shared with us.

We were relieved, of course, that we didn't see a bump in mortality in the first six months post-go-live; and we were actually gratified that we started seeing a decrease in mortality shortly after go-live, but that this lower rate continued. So 15 months after go-live was in 2009, and we said, we should really share our experience. Eventually, we ended up with 18 months of data showing that our mortality was dropping, so at that point, we decided we should write this up. And the various articles have built on each others' findings. Pittsburgh Children's published in 2005; Seattle Children's published their no-mortality-increase data in late 2006; we went live in 2007; and we published our data in 2010.

HCI: To what do you essentially attribute your decrease in mortality?

Longhurst: I'd point you to another important study that came out a month before us-it was by authored Dr. David Classen in Health Affairs, regarding the Leapfrog results at 64 different hospitals. And one of the reasons that Paul [Sharek] and I think it's important is that it showed that at 65 different hospitals, the efficacy in preventing medication errors within CPOE varied very dramatically; and the top-performing hospitals had all different vendors. So I think it very clearly shows what many of us had known before, and that it's not the software, it's how you implement it.

IT'S NOT POSSIBLE TO FOLLOW PRE-PACKAGED, SHORT-TIME-SPAN, COOKIE-CUTTER METHODOLOGIES. DO IT THOUGHTFULLY AND METHODICALLY.

So we believe our decrease had to do with our decision-making. We tried to make very thoughtful, evidence-based decisions at every step along the way. Furthermore, I mentioned one of our other board indicators, and the discussion section of the Pediatrics article points to some hypotheses around this associated change. And one of those hypotheses is that a lot of people point to medications as a reason to implement CPOE, right? But we didn't see a statistically significant change in medication harm; we already had a very low rate of medication harm, based on our previous use of paper-based order sets. And although medication harm should never happen, it's exceedingly rare in children's hospitals.

So the 20-percent decrease in mortality-other studies have pointed CPOE impacting turnaround times. There's a classic study from 2002 about Ohio State University that showed that they had a statistically significant decrease in turnaround times for medications for radiology orders. We read that study and completely replicated it at our hospital. So to measure from time of order to time of lab or radiology result, in the CPOE world, you can get that automatically out of the system; but on paper, you have to do time and motion studies. And we saw statistically significant improvement in turnaround times for stat medication orders, stat radiology orders, and stat laboratory orders.

We speculate that one of the potential reasons for this association is the improvement in turnaround times. Because if you go back to the Pittsburgh Children's study, experts believe one of the reasons for their temporary increase in mortality is that they experienced increased their turnaround times. They only looked at transport patients in their study; and previously, they had been able to write orders prior to the arrival of the patients. After go-live, they had to have the patients in-house physically before doing the CPOE orders. That probably impacted mortality among certain patients. So that relates to one of the main two hypotheses that Paul and I had.

That's a huge unheralded benefit of CPOE. We didn't make the lab run any faster or the pharmacy move any faster. We eliminated the front-end waste in wait times, so that decreases the amount of time between when I as a physician order something and the time when the task is completed on behalf of the patient.

And the second main hypothesis has to do with simple access to information; and this is something that's difficult to quantify. But we know that when nursing documentation sits at the bedside, it only gets looked at by physicians when they're at the bedside. Now, that information is pushed to them; and the simple fact of that availability to physicians may improve decision-making in terms of treatment, and more frequent review of information.

HCI: What would your advice to CIOs and CMIOs be, based on the lessons learned from your CPOE implementation experience?

Longhurst: My advice is, number one, CPOE is the most massive process change-based disruption that you can introduce into the healthcare process. So therefore, it needs to be introduced with as much gravitas and thought as possible, and with as much evidence as possible. In other words, it's not possible to follow pre-packaged, short-time-span, cookie-cutter methodologies. That's one big takeaway point: do it thoughtfully and methodically. And my personal suggestion is that people overlook the 20 years of peer-reviewed information in the clinical literature. There's all this good, published research. And it's important to have clinical leaders be familiar with that research. We literally had a monthly sharing of clips where everyone involved could read that information. And just as we strive to practice evidence-based care, we should strive to practice evidence-based EMR implementations.

Healthcare Informatics 2011 June;28(6):76-78