Skip to content Skip to navigation

Shifting from Predicting to Forecasting: One Healthcare Expert’s Perspective

August 3, 2014
by Mark Hagland
| Reprints
Though predicting has been the norm among healthcare industry observers for many decades, Jeffrey C. Bauer, Ph.D. says it’s time to make the shift to forecasting going forward

Though predicting has been the norm among healthcare industry observers for many decades, one  industry expert says it’s time to stop predicting and begin forecasting in earnest. Why? In a dynamic and rapidly changing industry like healthcare, the limitations of prediction-making are becoming more and more obvious, says Jeffrey C. Bauer, Ph.D., an independent economist and health futurist who has been a medical school professor, consultant, speaker, and writer.

The Chicago-based Bauer has written several books and over 250 articles. His new book, Upgrading Leadership’s Crystal Ball: Five Reasons Why Forecasting Must Replace Predicting and How to Make the Strategic Change in Business and Public Policy (CRC Press/Taylor & Francis Group), has just been released this summer. As Bauer notes on the book’s jacket, “Although predicting and forecasting are usually used as synonyms for a single process, they are conceptually and methodologically quite different… Readers will learn the real-world value of distinguishing between predicting (extrapolating historical trends) and forecasting (estimating the probabilities of possibilities).

Jeffrey C. Bauer, Ph.D.

Bauer’s new book follows in the wake of a number of healthcare-specific books, including Statistical Analysis for Health Care Decision-Makers. (Back in 2007, he co-authored Paradox and Imperatives in Health Care: How Efficiency, effectiveness, and E-Transformation Can Conquer Waste and Optimize Quality, with Mark Hagland, who is now HCI Editor-in-Chief.) Hagland spoke recently with Bauer about his new book, and the implications for healthcare leaders of principles of forecasting. Below are excerpts from that interview.

You articulate it fully in the book, but for those who have not yet read the book, can you succinctly articulate the difference between prediction and forecasting, and why that difference matters?

Prediction is an estimate of a single value at a future time, such as, healthcare will account for this percentage of the GDP, or we’ll be short this many doctors, or there will be this much construction activity; whereas forecasting accepts the possibility of A, B, C, and D, as opposed to just A. And rather than any specific probabilities, it gives you values. In the book, I talk about the prediction of the percentage of GDP spent on healthcare; my current understanding is that healthcare is declining as a percentage of GDP. So I put about a 20-percent probability that overall healthcare spending as a percentage of GDP would rise from where it is today. I think there’s a 60-percent probability that healthcare will actually decline as a percentage of GDP. Wall Street believed that healthcare spending would reach 20 percent of GDP, and as recently as Monday of this week, the Wall Street Journal declared healthcare to be the great growth industry based on all the newly insured. But as a forecaster, I would say there’s a 60 percent change that healthcare will become a declining industry in the coming years. The current accepted prediction is that healthcare should be at 18 percent this year, but right now it’s at 17 or 17.2 percent, by many estimates.

Furthermore, predictions have never been right. And I note in the book that not a single prediction from the government has ever been right in the past 40 years.

You describe the “five fatal flaws of predicting”: discontinuities in system dynamics; violations of model assumptions; deficiencies of available data; failures of previous predictions; diversions from strategic innovation. What should our audience know about these flaws that are inherent in prediction activity?

First of all, they should know that predictions have never been right in healthcare. That’s the number-one reason not to pay attention to predictions. Now, why have they never been right? First of all, the data are rotten to the core. Our databases are terrible; so big data is laughable in healthcare, because we’ve got all these numbers that are neither reliable nor valid. And so predictions are never right, because the data are not at all representative of what’s happening. And then the models used have very strict requirements related to the data distribution, that almost never hold in healthcare. Healthcare data almost never come to look like the bell-shaped curve, which is a core assumption in predictive modeling. And even if you have good data and good models, if the underlying circumstances that explain the data and models, you’ll not be successful.

Essentially, your perspective is that healthcare is simply too dynamic right now, correct/

Yes, that’s right, things are simply changing too fast. The underlying concepts of how clinicians should be treating patients—the clinical knowledge is changing too fast now.  There’s a change in the clinical paradigm taking place right now; and all of our data represents 20th-century acute-care-based medicine. But as 21st-century medicine moves forward, cures will eventually become largely irrelevant; it will be about forecasting what might happen, and managing the possible outcomes. We’re really at the point where the 20th-century clinical paradigm, involving waiting until a disease manifests, is quickly moving out the window. It’s now about managing disease, such as HIV, or musculoskeletal diseases, or preventing heart attacks.

Everything is moving towards a chronic care management platform.

Yes, though I prefer to use the term disease management.