Study: Specific EHR Vendors Associated with Better Hospital MU Performance | Healthcare Informatics Magazine | Health IT | Information Technology Skip to content Skip to navigation

Study: Specific EHR Vendors Associated with Better Hospital MU Performance

November 27, 2017
by Heather Landi
| Reprints
Click To View Gallery

A recent study suggests that all electronic health record (EHR) systems may not be created equal as it relates to hospitals’ performance on Meaningful Use criteria. Certain vendors are more often associated with better Meaningful Use performance than others, according to a study published in the Journal of the American Medical Informatics Association.

The study, by Jay Holmgren of Harvard Business School, Julia Adler-Milstein, of the University of California San Francisco, School of Medicine, and Jeffery McCullough of the University of Michigan School of Public Health, assessed whether there was variation across EHR vendors in the degree to which hospitals using products from those vendors were able to achieve high levels of performance on Meaningful Use criteria.

As the study points out, the Medicare and Medicaid Electronic Health Record (EHR) Incentive Programs have sought to drive widespread EHR adoption among doctors and hospitals to realize quality and efficiency benefits. The programs were designed with two coordinated components: EHR Meaningful Use criteria to which providers and hospitals attest, and EHR certification criteria that vendors must meet to ensure that their products allow providers/hospitals to meet the Meaningful Use criteria. Prior research revealed that the majority of providers and hospitals have adopted a certified EHR and attested to Meaningful Use criteria. Less is known about variation across certified products in supporting providers’ ability to use EHRs, the researchers wrote.

The researchers also point out that prior studies have revealed variation in the quality of key capabilities of certified systems and the certification process itself has been criticized for only requiring that vendors meet a constrained set of functions that are known to them in advance.

For the study, the researchers created a cross-sectional national hospital sample from the Office of the National Coordinator for Health Information Technology (ONC) EHR Products Used for Meaningful Use Attestation public use file and the Centers for Medicare & Medicaid Services Medicare (CMS) EHR Incentive Program Eligible Hospitals public use file. The researchers used regression models to assess the relationship between vendor and hospital performance on six Stage 2 Meaningful Use criteria, controlling for hospital characteristics. And, the researchers also calculated how much variation in performance is explained by vendor choice.

The study results indicate that three hospitals involved in the study were in the top performance quartile for all Meaningful Use criteria, and all used Epic. Of the 17 hospitals in the top performance quartile for five criteria, 15 used Epic, one used MEDITECH, and one used a smaller vendor. For the 68 hospitals in the top quartile for four criteria, 44 (64.7 percent) used Epic, 8 (11.8 percent) used Cerner, and 6 (8.8 percent) used MEDITECH.  For the 355 hospitals that were not in the top quartile of any criteria, 102 (28.7 percent) used MEDITECH, 89 (25.1 percent) used McKesson, 72 (20.3 percent) used Cerner, 51 (14.4 percent) used MEDHOST, and 24 (6.8 percent) used Epic.

The researchers found significant associations between specific vendors and level of hospital performance for all six Meaningful Use criteria. “Epic was associated with significantly higher performance on five of the six criteria; relationships for other vendors were mixed, with some associated with significantly worse performance on multiple criteria,” the study authors wrote. EHR vendor choice accounted for between 7 percent and 34 percent of performance variation across the six criteria, according to the study authors.

“In addition, the specific vendor appears to matter, with some vendors associated with significantly better or worse performance across multiple criteria. This suggests that hospitals may need to take vendor choice into account if they seek higher achievement on meaningful use criteria and cannot rely solely on the EHR certification process to ensure that all systems are equally capable,” the study authors wrote.

The fact the one vendor was significantly positively associated with multiple criteria while others reflected a mix indicates that vendors are making different design choices, and indicates that some vendors are simply doing a better job than others at creating systems that enable hospitals to achieve high levels of performance on meaningful use criteria, the study authors noted.

What’s more, the researchers wrote, “Our results also suggest that policymakers should consider strengthening the certification program to minimize such vendor-to-vendor differences or take steps to improve transparency about this variation.”

The study authors contend that the study findings have important implications for both policymakers and hospital leaders. “While all hospitals in our sample met Stage 2 meaningful use requirements, there was significant variation in achievement across certified vendors. This suggests that the current EHR certification process does not achieve a key intended goal of giving providers confidence that they are buying a ‘good’ EHR. This may be due to the fact that the certification process only tests a narrowly constrained set of ‘technical; meaningful use requirements and does so in a controlled laboratory setting,” the researchers wrote.

The study authors suggest that policymakers consider including more real-world trials to assess how EHRs perform in simulated clinical events that are not revealed to vendors beforehand. And, the study results may also indicate that the quality of vendor support services can impact Meaningful Use performance. “Policymakers may therefore wish to explore a certification process that includes an assessment of the quality of these services that holistically evaluates software and vendor support services,” the researchers wrote.

For hospital leaders, the study underscores the need to make careful choices when selecting an EHR vendor and offers some guidance on which products may be stronger, the study authors stated. However, given that vendor choice explained no more than one-third of performance variation, success may not be as simple as picking the “best” vendor. “Additional evidence revealing that hospitals have achieved disparate quality results using similar software systems further suggests the importance of organizational factors such as implementation and vendor services, IT staffing and support, training programs, and commitment to optimizing EHR use, which are likely more important to achieving high levels of performance,” the study authors wrote.

Get the latest information on Health IT and attend other valuable sessions at this two-day Summit providing healthcare leaders with educational content, insightful debate and dialogue on the future of healthcare and technology.

Learn More



ONC Roundup: Senior Leadership Changes Spark Questions

The Office of the National Coordinator for Health IT (ONC) has continued to experience changes within its upper leadership, leading some folks to again ponder what the health IT agency’s role will be moving forward.

Media Report: Walmart Hires Former Humana Executive to Run Health Unit

Reigniting speculation that Walmart and insurer Humana are exploring ways to forge a closer partnership, Walmart Inc. has hired a Humana veteran to run its health care business, according to a report from Bloomberg.

Value-Based Care Shift Has Halted, Study Finds

A new study of 451 physicians and health plan executives suggests that progress toward value-based care has stalled. In fact, it may have even taken a step backward over the past year, the research revealed.

Study: EHRs Tied with Lower Hospital Mortality, But Only After Systems Have Matured

Over the past decade, there has been significant national investment in electronic health record (EHR) systems at U.S. hospitals, which was expected to result in improved quality and efficiency of care. However, evidence linking EHR adoption to better care is mixed, according to medical researchers.

Nursing Notes Can Help Predict ICU Survival, Study Finds

Researchers at the University of Waterloo in Ontario have found that sentiments in healthcare providers’ nursing notes can be good indicators of whether intensive care unit (ICU) patients will survive.

Health Catalyst Completes Acquisition of HIE Technology Company Medicity

Salt Lake City-based Health Catalyst, a data analytics company, has completed its acquisition of Medicity, a developer of health information exchange (HIE) technology, and the deal adds data exchange capabilities to Health Catalyst’s data, analytics and decision support solutions.