In 2014, University of Miami (UM) Health System and Lockheed Martin, a Bethesda, Md.-based global security and aerospace company with involvement in healthcare analytics, announced a multi-disciplinary partnership with the end goal to access patient data faster to allow for more preventive healthcare. According to David Seo, M.D., UM Health System’s chief research information officer and chief medical informatics officer (CMIO) of the Miller School of Medicine at the time of the announcement, the plan was to help clinical leaders “come up with actionable data that is truly important for the patient the physician is taking care of.”
At the time, Seo additionally noted, “This information should not just be about the patient's past. We need a data environment that can do complex statistical analysis to help us move away from reactive medicine and toward proactive medicine, in which we get to patients before they get sick and prevent the disease from occurring." Indeed, prior to this announcement, in 2013, the Lockheed Martin/UM Health System team established a data environment, implemented big data analytics and predictive modeling tools, and started to stratify patient data and conduct risk assessments.
David Seo, M.D.
It was a few years ago when Seo, now associate vice president, information technology for clinical applications and still CMIO at UM Health System, said he and other health IT leaders at the organization began to realize the evolution of where healthcare was going. “Patient-centered medical homes and ACOs [accountable care organizations] were the trends under the main idea of managing risk,” Seo says in a more recent interview with Healthcare Informatics. “I was getting multiple calls and visits from vendors offering analytics solutions, one after the other, and what became clear was they were not offering a true full suite of what a health system needs to manage risk. Our own EHR [electronic health record] vendor talked to us, but even what they could provide was limited. We knew were headed towards a clinically integrated network and other things of that nature. We needed a company that had a long track record of understanding data analytics and security,” Seo says, referencing the partnership with Lockheed Martin.
Building from the Ground Up
Seo readily acknowledges that predictive analytics in healthcare “is still very much in its infancy no matter who you talk to.” Indeed, aside from the basics such as readmissions, true predictive analytics has not come to fruition, he notes. To this end, University of Miami started out with a diabetes risk model, and clinician leaders have shown that the model can fit within providers’ workflows, Seo says. He adds that the risk model can be ordered through the organization’s order entry system, or it can have a patient ask to run that risk model themselves in test environments. “The risk model returns a score, so you understand your risk of developing diabetes over the next five years, for example. And now we are engaged with our clinical staff to [look at] things such as what is the threshold we would set to apply an intervention, for instance,” Seo says.
Seo further emphasizes the importance of the health system’s work around different validations, which he says is a necessity before a risk model of this scope goes into production. He explains two key areas around validations. First, the validation of a phenotype or a diagnosis using EHR data needs to be validated for the system. “If I am going to say you do or do not have diabetes for example, that needs to be valid, and you need to understand what the positive predicted value of that phenotype is,” he says.
Second, he says, the diabetes prediction needs to be valid for a specific population. “I like to say that population health will be local, so the diabetes model that we pull from the literature has been validated form a highly specialized population that perhaps is of different racial or ethnic origins from our south Florida population. So what we’re doing is validating the phenotype in our population, and also understanding what the performance of that model is in our population. These are two important steps before going live with this prediction model,” Seo says.
Seo adds that even though health IT leaders may think everything should work exactly as intended, they still need to go back and look at a statistically significant number of patients, by literally going into their charts and confirming that the phenotype or diagnosis that they declared using EHR data is actually relevant in real life. “When you’re dealing with big data, data quality, and missing variables, these things all come into play, so you need to make sure your starting points and basic assumptions are correct. It’s a necessity for using EHR data for predictions,” he says.
When talking about predictive models and big data approaches, Seo feels that validation is the leap that is not really thought about or considered among clinical folks. “It’s what our organization has learned,” he says. “In theory, it’s excellent to say that I can predict this or that, when in practical reality, if you’re using this data to clinically treat patients, there needs to be granular additional steps of validating data, validating your phenotypes, and validating your predictions. There is a large amount of work that needs to go in to make sure that what you’re doing is good for patients,” he says.
Navigating in a New Era