On its Mission to Capture Unstructured EHR Data, Mercy Leaders Realize the Value of NLP | Healthcare Informatics Magazine | Health IT | Information Technology Skip to content Skip to navigation

On its Mission to Capture Unstructured EHR Data, Mercy Leaders Realize the Value of NLP

April 10, 2018
by Rajiv Leventhal
| Reprints
One M.D. involved in the project says that the initiative has restored his faith in “NLP’s ability to get us out of this data capturing conundrum”

One of the core issues that clinicians and others have had with EHRs (electronic health records) is that so much of the relevant medical information that’s vital to improving patient care is “trapped” within EHR physician notes. As such, patient care organizations are more and more turning to natural language processing (NLP)—a technology that allows providers to gather and analyze unstructured data, such as free-text notes.

One such health system that has worked to leverage NLP to exact and transform data collected during routine clinical care is the St. Louis-based Mercy, one of the country’s largest health systems that includes more than 40 acute care and specialty hospitals, and 800 physician practices and outpatient facilities. In Mercy’s submission for the 2018 Healthcare Informatics Innovator Awards Program, organizational leaders outlined the key details behind the health system’s project, called “Using NLP on EHR notes in Heart Failure Patients.” The submission ended up receiving semifinalist status in this year’s program.

The overarching goal of this initiative, says Kerry Bommarito, manager of data science at Mercy, was to use NLP to extract key cardiology measures from physician and other clinical notes, and then incorporate the results into a dataset with discrete data fields. The dataset would then be used to obtain actionable information and contribute to the evaluation of outcomes of medical devices in heart failure patients—a subset population of which there have been approximately 100,000 patients in the Mercy system going back to 2011.

Bommarito explains that three core measures that are commonly stored in clinical notes and not available in discrete fields include ejection fraction measurement, patient symptoms including dyspnea (breathing difficulty), fatigue and dizziness, and the New York Heart Association (NYHA) heart failure classification—the latter which places patients in one of four categories based on how limited they are during physical activity.

“To be able to best classify how severe the CHF [congestive heart failure] was, we really needed to get these measures out of the physician notes,” Bommarito attests, adding that since heart failure is a chronic and progressive syndrome, changes in these three measures are important indicators of heart failure decompensation.


Driving Success at Regional Health: Approaches and Challenges to Optimizing and Utilizing Real-Time Support

Regional Health knew providing leading EHR technology was not the only factor to be considered when looking to achieve successful adoption, clinician and patient satisfaction, and ultimately value...

Kerry Bommarito

Joseph Drozda, M.D., cardiologist, director of outcomes research at Mercy, says that “perhaps 60 percent of the data you would really like is available to you as discrete data in the EHR. The remaining 40 percent is contained in text and clinical notes, and in order to get the meaningful data out, you have to [use] something like NLP to capture it.”

Indeed, Dr. Drozda believes the issue stems from most EHRs being originally developed as billing systems that were designed to capture data necessary for populating a claim form and submitting a bill. “They weren’t really designed for clinical care; that came afterwards,” he says. “And in a lot of ways, in the early stages, the EHR systems were putting the paper records in electronic format without any regard to trying to use the data on the back end. It was pretty much all text. I think it was a basic design flaw in most EHR systems right from the beginning, though we are starting to overcome it.”

Nonetheless, Drozda notes that this challenge is still difficult to overcome, because in order to capture something like ejection fraction—a measurement in determining how well the heart is pumping out blood, helping to diagnose and track heart failure—visually speaking, a clinician has to look for a dropdown menu or find someplace to enter a value, which takes extra time. “So there are two things working against you—the basic underlying technology challenges and the workflow challenges that clinicians face in entering discrete data. We have to be very careful in how much discrete data [we make] clinicians enter, as they are already concerned with deaths by 1,000 clicks,” he says.

Joseph Drozda, M.D.

As such, for Mercy’s heart failure patient population, project leaders brought in all of the notes that they had at the time, totaling about 34 million going back seven years. NLP queries were developed by a team of Mercy data scientists to search for relevant linguistic patterns, and then the queries were evaluated for both precision and recall. When the queries were determined to have a high accuracy, the results were integrated into a comprehensive data schema that contains real-world clinical data for each heart failure patient from before the diagnosis of CHF to their current state, Bommarito explains.

The final queries were validated and had a high accuracy with an F-measure (the measure of as test’s accuracy) score above 0.90 (1 is the highest F-measure value possible). “This F-measure score shows that Mercy’s queries were highly precise (positive predictive value) and had high recall (also known as the true positive rate or sensitivity), says Bommarito. What’s more, “These results show that natural language processing is a reliable and accurate method to extract relevant data from clinical notes in a CHF population,” she concluded.

Mercy leaders further note that capturing this data will help them answer a key clinical question: how does cardiac resynchronization therapy (CRT) affect patients who have heart failure? As they explain, CRT is a way of pacing the heart so that it pumps more efficiently. And it is used in patients not just with heart failure, but with a specific type: heart failure with reduced ejection fraction. “So when we are looking at how CRT affects the natural history of patients with heart failure with reduced ejection fraction, we need to have the ejection fraction and the NYHA functional classification that tells us how the patient is doing symptomatically,” Drozda explains.

At that point, Mercy can capture its entire population of heart failure patients, which includes reduced ejection fraction, and those who have reserved ejection fraction—but not reduced—by using NLP. “And we can look at these patients long before they receive CRT; we are going back three years in history to get started, so we can see how the patient did for the year or two before the CRT device was put in, and see how he or she did after it was put in, and then compare those patients with CRT devices with those who didn’t [have them]. So it’s a great opportunity to look at the impact of a high-end technology on a very sick population of patients with heart failure. And without NLP, we couldn’t do this,” Drozda contends.

What’s more, the NLP software that Mercy leveraged, from U.K-headquartered Linguamatics, included a library of terms that was a great starting point for the project’s team, Bommarito says. She offers an example that when Mercy was looking for a shortness of breath symptom, for instance, the library had all of the different medical ways that a clinician might state ‘shortness of breath.’ So we didn’t have to sit there and think of ways that the doctor might say ‘shortness of breath.’ And that was really helpful.’”

Adds Drozda, “NLP has come a long way. I have been a skeptic based on others’ experiences, but this is my first time involved and the results [we have gotten] have been tremendous. Those F-measure scores are amazing to me; it’s restored my faith in NLP’s ability to get us out of this data capturing conundrum.”

The Health IT Summits gather 250+ healthcare leaders in cities across the U.S. to present important new insights, collaborate on ideas, and to have a little fun - Find a Summit Near You!


VA, Cerner Leaders Detail Progress on EHR Implementation, Interoperability Efforts

November 14, 2018
by Heather Landi, Associate Editor
| Reprints
Click To View Gallery

The U.S. Department of Veterans Affairs (VA)’s new $16-billion-dollar Cerner electronic health record (EHR) system will use open application programming interface (APIs) and Fast Healthcare Interoperability Resources (FHIR) standards to enable interoperability with the private sector, according to a Cerner executive, which potentially positions the VA as a leading force to drive interoperability forward in the healthcare industry.

The ability of the VA’s healthcare system to seamlessly share patient data with the U.S. Department of Defense (DoD) as well as health systems and physicians in the private sector continues to be a top concern among Congressional leaders as the VA is now six months in to its implementation of a new Cerner EHR, and the topic dominated a House oversight subcommittee hearing on Wednesday.

Congressional leaders pointed out that interoperability between VA and DoD and between VA and community providers would be key to the success of the VA electronic health record (EHR) modernization effort. “If you can’t make that step work, then this won’t work,” Rep. Phil Roe, M.D., (R-Tenn.) chairman of the House Veterans Affairs committee, said.

During the hearing, members of the House Veterans Affairs' technology modernization subcommittee reviewed the electronic health record modernization (EHRM) program’s accomplishments, to date, and questioned VA and Cerner leaders about implementation planning, strategic alignment with the DoD’s MHS Genesis project, as DoD also is rolling out a new Cerner EHR, as well as interoperability efforts.  

The VA signed its $10 billion contract with Cerner in May to replace VA’s 40-year-old legacy health information system—the Veterans Health Information Systems and Technology Architecture (VistA)—over the next 10 years with the new Cerner system, which is in the pilot phase at DoD. The VA project will begin with a set of test sites in the Pacific Northwest in March 2020.


Driving Success at Regional Health: Approaches and Challenges to Optimizing and Utilizing Real-Time Support

Regional Health knew providing leading EHR technology was not the only factor to be considered when looking to achieve successful adoption, clinician and patient satisfaction, and ultimately value...

In October, the U.S. Secretaries of the VA and DoD signaled their commitment to achieving interoperability between the two agencies by implementing a single, seamlessly integrated EHR, according to a joint statement both agencies issued. VA Secretary Robert Wilkie and Defense Secretary James N. Mattis signed a joint statement Sept. 26 pledging that their two departments will “align their plans, strategies and structures as they roll out a EHR system that will allow VA and DoD to share patient data seamlessly."

However, subcommittee chairman Jim Banks (R-Ind.) noted during the hearing, “Community provider interoperability has always been the elephant in the room. VA-DoD interoperability is very important, but VA is further behind in exchanging records with its community partners. There are helpful tools, such as health information exchanges (HIEs), but no out-of-the-box EHR system completely solves this problem.” Banks added, “Community interoperability is a very real problem, and for $16 billion, VA had better solve it.”

It is estimated that up to a third of VA patients receive care in the private sector.

"I’m not ready to sound the alarm, but I’ve heard very little on the subject [interoperability with DoD and community providers],” Banks said, noting that a review by industry experts indicated that VA and DoD need to be on the same instance of the Cerner EHR in order to achieve seamless interoperability. “That means both departments have to pull patient data from the same database. The two implementations have to be joined at the hip. It raises the stakes. It’s important to put this reality out in the open, and early.”

During her testimony, Laura Kroupa, M.D., acting chief medical officer with the VA’s Office of Electronic Health Record Modernization (OEHRM), noted that interoperability with community healthcare providers was a challenge that VA and Cerner leaders were working together to address.  “Going on the Cerner platform will allow us to utilize national systems in place for interoperability. Our community care councils also look at all the different workflows for how patients get referred into and out of the VA to make sure that information is exchanged and put into the system, not just as a piece of paper or image, but actually the data itself,” she said. Kroupa said project leaders are working to utilize interoperability mechanisms that Cerner currently has as well as HIE initiatives already in place, such as Carequality and CommonWell, to ensure interoperability between VA and the private sector.

John Windom, who leads the VA’s EHR modernization project as the executive director of VA’s OEHRM program, said, “There are two issues—one is technology-based which is solved, the HIEs, CommonWell, Carequality, allow seamless exchange of information. But, there is also another piece, the information has to be put in, and so that information has to be made accessible by the people on those networks; we’ve got the technology piece solved.”

Travis Dalton, president of government services at Cerner, testified that Cerner would use open APIs and FHIR-based integration to enable interoperability between VA and healthcare providers in the private sector.

“We’ve committed to that contractually. It’s going to happen, it’s technically possible and feasible,” Dalton said, adding, “What will be powerful to the industry and commercial partners is if VA and DoD choose a common standard. That will move the industry forward because this isn’t always a technical issue, it’s a standard-based issue. The power of the DoD and VA to make that choice to move it forward will influence the commercial marketplaces. The tools exist, through HIE and Direct exchange; it’s a standards issue.”

Congress created the technology modernization comittee to provide more rigorous oversight of the project amid concerns about the project’s cost and alignment with the defense department’s electronic health record roll-out.

There have been ongoing questions about VA leadership, specifically with regard to the EHR modernization project, beginning with the ouster of the previous VA Secretary, David Shulkin, M.D., earlier this year, as well as other shake-ups, including the resignation of Genevieve Morris only two months after she was tapped to lead the VA’s EHR project.

An investigation by ProPublica, detailed in a report published Nov. 1, asserts that VA’s EHR contract with Cerner has been plagued by multiple roadblocks during the past year, including personnel issues and changing expectations. According to that report, Cerner rated its EHR project with the VA at alert level "yellow trending towards red.” To investigate the underlying factors that have contributed to the EHR project's problems, the publication reviewed internal documents and conducted interviews with current and former VA officials, congressional staff and outside experts.

In parallel, Rep. Banks has expressed concerns with the VA’s “apparent loss of focus” on innovation, specifically as it relates to open APIs. In a letter to acting VA Deputy Secretary James Byrne dated Oct. 10, Banks noted that two years ago the VA initiated an open-API gateway interoperability platform concept, called Lighthouse. Back in March, during the HIMSS Conference, the VA also announced an open API pledge, with the launch of a “beta” version of its Lighthouse Lab, which offers software developers access to tools for creating mobile and web applications to help veterans better manage their care, services and benefits. Banks wrote that these efforts seem to have “lost momentum.”

In the letter, Banks noted that the VA needs a flexible platform to translate data coming in from multiple EHRs and on which to build, and so its private sector partners can build, interfaces to and from medical practice billing systems, insurance companies, external applications, veterans’ devices and one day Medicare and Tricare’s systems. “The need to ‘future-proof’ the technology that VA is acquiring is very real. Moving forward with the open-API gateway and sustaining the open API pledge are important steps to do that,” Banks wrote.

Progress Made in the First 180 Days

During the hearing, Windom outlined the EHRM program’s accomplishments, to date, including the establishment of 18 workflow councils and current state assessments of the initial implementation sites. VA and Cerner project leaders also completed an analysis report to assess the DoD’s MHS Genesis system as EHRM’s baseline. The workflow councils are mostly comprised of clinicians in the field who provide input to enable configuration of national standardized clinical and operational workflows for the VA's Cerner EHR system, Windom said.

Dalton said the site visits of the initial implementation sites provided important insights into VA’s IT needs. “VA has a unique patient population, you’ve got an older, sicker population, with unique needs, such as behavioral health. Some areas that we uncovered that we need to focus on now include telehealth, behavioral health and reporting. These are big content areas,” he said, adding, “I expect the work that we do will help to lead us into the future in that area. We expect that as we work closely together to meet the needs of the agency that will help to makes us better commercially.”

Dalton said the VA Cerner are committed to applying commercial best practices, as well as any lessons learned from our DoD experience, to the VA’s EHRM program.

“We learned some hard lessons with the DoD experience,” he said. “Transformation is always difficult. We’re doing things a lot differently—we’re engaging with sites early and often. We’re also doing more workshops up front, so it’s more of an iterative process.” And, he added, “This is a provider-led process. We have the 18 councils that are assisting us with validation of the workflow.”


More From Healthcare Informatics


Driving Success at Regional Health: Approaches and Challenges to Optimizing and Utilizing Real-Time Support

Tuesday, December 11, 2018 | 3:00 p.m. ET, 2:00 p.m. CT

Regional Health knew providing leading EHR technology was not the only factor to be considered when looking to achieve successful adoption, clinician and patient satisfaction, and ultimately value from their investment.

Providing their clinicians across two states with real-time support during and after go-live to drive utilization and ensure efficiency with the EHR was important to drive success. In addition, their patients needed support to understand the value of the Epic MyChart patient portal and how to access the system.

Related Insights For: EHR


EHR Usability Issues Impact Pediatric Patient Safety, Research Finds

November 13, 2018
by Rajiv Leventhal, Managing Editor
| Reprints

In an analysis of 9,000 pediatric patient safety reports from three healthcare organizations, researchers found that 36 percent of the reports were related to EHR (electronic health record) usability issues.

The research, published in the November issue of Health Affairs, and led by Raj Ratwani, Ph.D., director of the National Center for Human Factors in Healthcare, Washington, D.C.-based MedStar Health, and others, aimed to further understand the specific issues around why pediatric populations are uniquely vulnerable to the usability and safety challenges of EHRs particularly those related to medication.

To understand specific usability issues and medication errors in the care of children, the researchers analyzed 9,000 patient safety reports, over a five-year span, from three different healthcare institutions—two stand-alone pediatric institutions and one adult and pediatric institution that used Epic and Cerner EHRs (two institutions used Epic, and one used Cerner)—that were likely related to EHR use.

Of the 9,000 reports, 3,243 (36 percent) had a usability issue that contributed to the medication event, and 609 (18.8 percent) of the 3,243 might have resulted in patient harm, the researchers found.

“The general pattern of usability challenges and medication errors were the same across the three sites. The most common usability challenges were associated with system feedback and the visual display. The most common medication error was improper dosing,” the research revealed.

The researchers noted in the study that pediatric patients are uniquely vulnerable to EHR usability and safety challenges because of different physical characteristics, developmental issues, and dependence on parents and other care providers to prevent medical errors. For example, they offered, lower body weight and less developed immune systems make pediatric patients less able to tolerate even small errors in medication dosing or delays in care that could be a result of EHR usability and safety issues.

Although the Office of the National Coordinator for Health Information Technology (ONC) has policies to promote usability—such as requiring system developers to incorporate feedback from clinicians into software design and development and mandating the testing of twelve high-risk EHR functions that are primarily related to medication—the researchers noted that these policies have not made a distinction between adult and pediatric populations. However, the 21st Century Cures Act of 2016 requires ONC to establish new voluntary criteria unique to EHRs used in the care of children.

For this research, the 9,000 reports—3,000 from each site—were reviewed to verify whether the events were related to the EHR and medication; determine whether EHR usability contributed to the event and, if it did, identify what the specific usability challenge was; identify the type of medication error; and identify whether the event reached the patient.

Of the 9,000 patient safety event reports that were collected, 56 percent were confirmed as being related to both the EHR and medication. Of these 64 percent had a usability issue as a contributing factor to the safety event, which amounts to 36 percent of the total 9,000 reports analyzed.

Of the 3,243 reports (36 percent) that had usability as a contributing factor, 19 percent reached the patient. Of these, 33 percent did not cause harm and did not require monitoring, 18 percent required monitoring or an intervention to prevent harm, 3 percent resulted in temporary harm, and the consequence was unknown for 46 percent, the researchers revealed.

One example of a usability issue that caused some harm was when a when a physician ordered five times the recommended dose of a medication without receiving an alert from the EHR, although the prescribed dose was outside the recommended range. Both vendor design and development, as well as implementation and customization, may be contributing to the challenges associated with system feedback, the researchers stated.

“To address this systemic problem, vendors and providers should consider developing more comprehensive design guidelines and use generalizable tools to assess usability and safety. The Leapfrog [computerized provider order entry simulation] tool, which assesses clinical decision support functionality, is one example of the types of tools that could improve the safety of implemented EHR products,” they said.

The researchers concluded, “To better prevent usability-related medical errors, the ONC could include safety as part of the voluntary certification criteria of EHRs for use with children and implement usability-related measures to assess EHR performance. Vendors and providers should use rigorous test-case scenarios based on realistic clinician tasks. Finally, the Joint Commission should assess EHR safety as part of its hospital accreditation program. The implementation of approaches such as these is needed to reduce patient harm related to EHR use.”

See more on EHR

betebettipobetngsbahis bahis siteleringsbahis