Study: Specific EHR Vendors Associated with Better Hospital MU Performance | Healthcare Informatics Magazine | Health IT | Information Technology Skip to content Skip to navigation

Study: Specific EHR Vendors Associated with Better Hospital MU Performance

November 27, 2017
by Heather Landi
| Reprints
Click To View Gallery

A recent study suggests that all electronic health record (EHR) systems may not be created equal as it relates to hospitals’ performance on Meaningful Use criteria. Certain vendors are more often associated with better Meaningful Use performance than others, according to a study published in the Journal of the American Medical Informatics Association.

The study, by Jay Holmgren of Harvard Business School, Julia Adler-Milstein, of the University of California San Francisco, School of Medicine, and Jeffery McCullough of the University of Michigan School of Public Health, assessed whether there was variation across EHR vendors in the degree to which hospitals using products from those vendors were able to achieve high levels of performance on Meaningful Use criteria.

As the study points out, the Medicare and Medicaid Electronic Health Record (EHR) Incentive Programs have sought to drive widespread EHR adoption among doctors and hospitals to realize quality and efficiency benefits. The programs were designed with two coordinated components: EHR Meaningful Use criteria to which providers and hospitals attest, and EHR certification criteria that vendors must meet to ensure that their products allow providers/hospitals to meet the Meaningful Use criteria. Prior research revealed that the majority of providers and hospitals have adopted a certified EHR and attested to Meaningful Use criteria. Less is known about variation across certified products in supporting providers’ ability to use EHRs, the researchers wrote.

The researchers also point out that prior studies have revealed variation in the quality of key capabilities of certified systems and the certification process itself has been criticized for only requiring that vendors meet a constrained set of functions that are known to them in advance.

For the study, the researchers created a cross-sectional national hospital sample from the Office of the National Coordinator for Health Information Technology (ONC) EHR Products Used for Meaningful Use Attestation public use file and the Centers for Medicare & Medicaid Services Medicare (CMS) EHR Incentive Program Eligible Hospitals public use file. The researchers used regression models to assess the relationship between vendor and hospital performance on six Stage 2 Meaningful Use criteria, controlling for hospital characteristics. And, the researchers also calculated how much variation in performance is explained by vendor choice.

The study results indicate that three hospitals involved in the study were in the top performance quartile for all Meaningful Use criteria, and all used Epic. Of the 17 hospitals in the top performance quartile for five criteria, 15 used Epic, one used MEDITECH, and one used a smaller vendor. For the 68 hospitals in the top quartile for four criteria, 44 (64.7 percent) used Epic, 8 (11.8 percent) used Cerner, and 6 (8.8 percent) used MEDITECH.  For the 355 hospitals that were not in the top quartile of any criteria, 102 (28.7 percent) used MEDITECH, 89 (25.1 percent) used McKesson, 72 (20.3 percent) used Cerner, 51 (14.4 percent) used MEDHOST, and 24 (6.8 percent) used Epic.

The researchers found significant associations between specific vendors and level of hospital performance for all six Meaningful Use criteria. “Epic was associated with significantly higher performance on five of the six criteria; relationships for other vendors were mixed, with some associated with significantly worse performance on multiple criteria,” the study authors wrote. EHR vendor choice accounted for between 7 percent and 34 percent of performance variation across the six criteria, according to the study authors.

“In addition, the specific vendor appears to matter, with some vendors associated with significantly better or worse performance across multiple criteria. This suggests that hospitals may need to take vendor choice into account if they seek higher achievement on meaningful use criteria and cannot rely solely on the EHR certification process to ensure that all systems are equally capable,” the study authors wrote.

The fact the one vendor was significantly positively associated with multiple criteria while others reflected a mix indicates that vendors are making different design choices, and indicates that some vendors are simply doing a better job than others at creating systems that enable hospitals to achieve high levels of performance on meaningful use criteria, the study authors noted.

What’s more, the researchers wrote, “Our results also suggest that policymakers should consider strengthening the certification program to minimize such vendor-to-vendor differences or take steps to improve transparency about this variation.”

The study authors contend that the study findings have important implications for both policymakers and hospital leaders. “While all hospitals in our sample met Stage 2 meaningful use requirements, there was significant variation in achievement across certified vendors. This suggests that the current EHR certification process does not achieve a key intended goal of giving providers confidence that they are buying a ‘good’ EHR. This may be due to the fact that the certification process only tests a narrowly constrained set of ‘technical; meaningful use requirements and does so in a controlled laboratory setting,” the researchers wrote.

The study authors suggest that policymakers consider including more real-world trials to assess how EHRs perform in simulated clinical events that are not revealed to vendors beforehand. And, the study results may also indicate that the quality of vendor support services can impact Meaningful Use performance. “Policymakers may therefore wish to explore a certification process that includes an assessment of the quality of these services that holistically evaluates software and vendor support services,” the researchers wrote.

For hospital leaders, the study underscores the need to make careful choices when selecting an EHR vendor and offers some guidance on which products may be stronger, the study authors stated. However, given that vendor choice explained no more than one-third of performance variation, success may not be as simple as picking the “best” vendor. “Additional evidence revealing that hospitals have achieved disparate quality results using similar software systems further suggests the importance of organizational factors such as implementation and vendor services, IT staffing and support, training programs, and commitment to optimizing EHR use, which are likely more important to achieving high levels of performance,” the study authors wrote.

Get the latest information on Health IT and attend other valuable sessions at this two-day Summit providing healthcare leaders with educational content, insightful debate and dialogue on the future of healthcare and technology.

Learn More



Survey: Infrastructure, Interoperability Key Barriers to Global HIT Development

A new survey report from Black Book Research on global healthcare IT adoption and records systems connectivity finds nations in various phases of regional electronic health record (EHR) adoption. The survey results also reveal rapidly advancing opportunities for U.S.-based and local technology vendors.

Penn Medicine Opens Up Telehealth Hub

Philadelphia-based Penn Medicine has opened its Center for Connected Care to centralize the health system’s telemedicine activities.

Roche to Pay $1.9B for Flatiron Health

Switzerland-based pharmaceutical company Roche has agreed to pay $1.9 billion to buy New York-based Flatiron Health Inc., which has both an oncology EHR and data analytics platform.

Financial Exec Survey: Interoperability Key Obstacle to Value-Based Payment Models

Momentum continues to grow for value-based care as nearly three-quarters of healthcare executives report their organizations have achieved positive financial results from value-based payment programs, to date, according to a new study from the Healthcare Financial Management Association (HFMA).

Cerner, Children's National to Help UAE Pediatric Center with Health IT

Al Jalila Children's Specialty Hospital, the only pediatric hospital in the United Arab Emirates, has entered into an agreement with Washington, D.C.-based Children's National Health System to form a health IT strategic partnership.

Telemedicine Association Names New CEO

The American Telemedicine Association (ATA) has named Ann Mond Johnson its new CEO, replacing Jon Linkous who stepped down suddenly last August after 24 years as the organization’s CEO.