Skip to content Skip to navigation

Exclusive Interview: Study Author Responds to Mostashari’s Critique; Talks Results

March 14, 2012
by Gabriel Perna
| Reprints
Danny McCormick, lead author of the study critiqued by ONC Chief, talks in depth about its findings

Not everyone’s study gets attention from Farzad Mostashari, M.D., the national coordinator for health IT. However, that’s exactly the situation Danny McCormick, M.D., an assistant professor of medicine at Harvard Medical School, and his team of researchers from the Cambridge Health Alliance and the CUNY School of Public Health found themselves in recently. The study appeared in the most recent edition of the journal Health Affairs last week and provoked a blog post response from Mostashari shortly after its release.

The study, with samplings from the National Ambulatory Medical Care Survey (NAMCS), examined 28,741 patient visits to 1,187 office-based physicians and concluded electronic access to computerized imaging results did not result in a decrease in the amount of imaging tests ordered nor the costs associated with those tests. In fact, the study found that physicians having access to those computerized imaging results may actually have lead to an increase in tests ordered and associated costs.

In his blog post, Mostashari said the authors of the study used data that was passé, from 2008, before the passage of the American Recovery and Reinvestment Act/ Health Information Technology for Economic and Clinical Health (ARRA/HITECH). He also said the authors did not “consider clinical decision support, which helps give providers the data tools they need to make appropriate care recommendations and the ability to exchange information electronically.”

In addition, Mostashari said the study was not designed to answer the question of cost, or associations between EHRs and quality. He added it did not consider variables like sicker patients and included a note about how a reduction in test orders is not the way to reduce costs regardless.

McCormick and his co-authors offered a rebuttal to Mostashari’s response, which they posted in a blog of their own. McCormick said the 2008 data is legitimate because the EHR vendor market is largely unchanged and the systems in place now are widely the ones used back then. He also said a lot of Mostashari’s suggestions to explain the study’s results, such as doctors who are inclined to order more tests are also inclined to purchase health IT for viewing test results electronically, are merely hypotheses, without proof.

Additionally, McCormick rejects Mostashari’s claim that the study failed to assess whether health IT improved the quality or appropriateness of care, noting that that wasn’t the purpose of the study. McCormick also noted Mostashari selectively touted a study that supported his claim, which he said was authored by members of the ONC, “without regard to study quality or statistical niceties.” There were several other refutations of Mostashari in the thoroughly-written blog post.

Few recent studies in healthcare IT that have driven this much controversy. HCI Assistant Editor Gabriel Perna recently got a chance to talk with McCormick about his findings. Here are excerpts from their conversation.

Explain the study’s background and what you were trying to accomplish.

I’m a primary care physician (PCP) myself, and back in 2005, I was on the Senate Health, Labor, Education & Pensions Committee under Senator [Edward] Kennedy and worked a fair amount of my time working on one of the precursor bills to the HITECH Act. Through that experience and our [McCormick and his team] experience as practicing clinicians, we had skepticism about the legitimacy of the long-made claim on the ability of health information technology to decrease test ordering and therefore costs throughout the U.S. Part of our question had to do with the fact most people put health IT forward as something that will bail out the spiraling cost spending crisis. We wondered if this was true.

We’ve used this federal database (NAMCS) for other research, not related to health IT. Several years ago,  it started collecting data on physicians use of health IT and so it was a natural idea to examine a large number of patient encounters across the U.S. and to ask the question, ‘Was the ability to see imaging results or the image itself associated with decreased test ordering?’ We thought most people would expect that.

What did you find?

Rather than finding there was a decrease or no change, we found in physicians’ offices that had the ability to see those electronic images or get the image reports, that image and blood test ordering was higher. If you included all imaging studies, having that electronic capacity to see the image results was associated with a 40 percent increase in test ordering. If you look specifically at the advanced imaging, i.e. CT and MRI scans, it was about a 70 percent increase.

One thing to note is this was not an experimental study where you take two populations and do one thing to one population and another to the other, and see the results. This was a cross-sectional study, where you’re collecting data and not treating one differently. Thus, you can’t say having the electronic capability caused the increase definitively. We avoided making any erroneous associations, and 90 percent of our analysis was doing multi-variant modeling to take patient-level differences, doctor differences, and practice differences into consideration. For instance, maybe a patient needs more imaging tests. Ninety percent of our analysis was to avoid making an erroneous conclusion based on something like that. We controlled for illnesses, characteristics, factors related to the doctors’ specialty, how they were paid, etc.