Skip to content Skip to navigation

Part 3: Blind Spots Can Kill Patients—Learning How to Avoid Errors

Printer-friendly version

In part one of this series (link: http://bit.ly/hCb0MP ), I introduced the widely accepted premise that delivering complete, accurate, up-to-date, and relevant information to care providers, surpassing today’s norms by using information technology, would lead to better care. In part two (link: http://bit.ly/hgjELo ), I revealed that dozens of types of cognitive errors, when carefully studied, represented a much larger barrier to care improvement than deficiencies in HIT. I also suggested that simply mandating CPOE, drug checks, ePrescribing, problem/medication/allergy lists, calculated and transmitted quality measures, patient engagement enhancers, care coordination improvements (clinical information exchange), and privacy and security protections (collectively Meaningful Use Objectives and Measures of Stage 1) would be unlikely impact these known cognitive errors. In some cases, such as the commission bias, streamlining information delivery, documentation and taking actions (CPOE and ePrescribing) could conceivably increase the error rate.

In part one of this series, I introduced the widely accepted premise that delivering complete, accurate, up-to-date, and relevant information to care providers, surpassing today’s norms by using information technology, would lead to better care. In part two, I revealed that dozens of types of cognitive errors, when carefully studied, represented a much larger barrier to care improvement than deficiencies in HIT. I also suggested that simply mandating CPOE, drug checks, ePrescribing, problem/medication/allergy lists, calculated and transmitted quality measures, patient engagement enhancers, care coordination improvements (clinical information exchange), and privacy and security protections (collectively Meaningful Use Objectives and Measures of Stage 1) would be unlikely impact these known cognitive errors. In some cases, such as the commission bias, streamlining information delivery, documentation and taking actions (CPOE and ePrescribing) could conceivably increase the error rate.
Coaching, one-on-one training, and feedback all necessary to reduce cognitive errors

Pages

Comments

Thanks Doc Benjamin. I see your point. It is possible to read the Romano and Stafford (link) article as an indictment, damaging the general public's perception of EHRs. And, that would be EHRs both With CDS and Without CDS:



I think smart readers and policy makers will turn that around.  What Romano and Stafford conclude is that simply having an EHR does not automatically confer improvement in quality indicators.  That's true, at least grossly according to their numbers shown above, compared both to paper records, and EHRs with or without CDS.

Their conclusion is substantially similar to my point in this blog series (i.e. cognitive processes need to be a central focus of our work), and the published work cited.  Installing HCIT doesn't deliver the goods.  To quote Clem McDonald's (link) comment


"Although EHRs without CDS may not improve adherence to clinicalguidelines, they are
(1) a necessary precondition for havingCDS (without electronic data, there can be no electronic supportfunctions);
(2) valuable for maintaining findable, sharable,legible, medical records; and
(3) when they are amply populated(ie, they contain at least 1 or 2 years of dictations, testresults, medications, and diagnoses/problems), physicans lovethem because there are no more lost charts or long waits onthe telephone for laboratory results. ..."


 As many have said before, "a hammer is not a house" and "an EHR is not an accountable, coordinated, evidence-based, care delivery system with patient engagement and dynamically managed processes."  That concept and a brilliant call to action was also provided this week by futurist Joe Flower in an article (link) I'll dub "Five Things.

Thanks again for your comment Doc Benjamin.  I share your observation that politicization with distortion (whether or not deliberate) about the role and value of EHRs can be damaging.  It's also very energy depleting.  Our job, in part through our comments here, are to shed clarity on real issues.  I am optimistic that those of us reading and writing in these blogs are putting this thinking to work, in our actions during our day jobs.



Jason,
You are welcome.   Thanks for joining in and providing a perspective.

This truly is a blind spot in informatics.  Those who have studied the diagnostic challenge and assessed it's impact on quality, cost, and satisfaction are always struck by the magnitude of the problem.  I agree with your statement that computers can be useful. I'll go further and acknowledge the requisite role of diagnostic CDS.

A couple of elaborations from my comments and Frank's.  There is an ONC Clinical Decision Support Workgroup that met last month in DC with over forty participants.  As I've mentioned previously, I participate in the EHRA which includes EHR vendors; we were capably represented by Jacob Reider (Allscripts), although other EHR vendors were there as well.  There were CDS-focused vendors including Logical Images, who, as you know, also focuses on the problem of diagnostic error, and, enterprise content providers Thompson-Reuters and Elsevier.  There were also vendors like Theradoc that focus on improving the therapeutic implications of an accurate diagnosis, which requires and leverages technology to validate and refine diagnoses.  Management, convening, harmonization and other leadership services were provided through AHRQ, NQF, Westat, and Rand folks.  (If I left you out, comment on this blog!) 
 
I would be remiss if I neglected to mention pioneering organizations like Partners, who sent Tonya Hongsermeier (clinical KM guru), Blackford Middleton, and David Bates (both extensively published in decades of CDS operational and safety benefit achievement).  I'll publish a link to the minutes when they become available to me; there were over twenty specific conclusions that resonate with your comments, Jason.  Volunteer power is nothing short of an ONC-produced miracle.

My professional experience, both as a clinician and a NIH/NLM-fellowship trained informatician (at the birthplace and home of DXplain) support your sober comments.  A clinician is not going to build an adequate differential diagnosis if she/he doesn't recognize the Livedo Reticularis rash, the P2 (heart sound), or knows to ask the contextually appropriate (and diagnostic) questions.  That can be a supra-human task, and the subject of an upcoming post originating from an article and dialogue with Dr. Graber, "Is this stuff teachable?"

It was a 1980s diagnostic CDS finding from AIRheum/LEARN, QMR and my work that all conclude "Get The Data" is distinct from dance steps two and three, "Find Action-Able Opportunities," and "Take Action."  Each is benefited by different kinds of computer support. 

I think we're in violent agreement, Jason!  Multiple distinct forms of CDS are required to improve care.



Great Post Joe! Great series as well.
The pioneering work of Drs Graber and Berner must be supported and encouraged, as it holds the promise of transforming the EMR from data storage to partner in care.

HCIT is truly in its infancy, and we must recognize that for many seasoned clinicians, the process of entering clinical information into a system is a trial all by itself. Actually being able to use the system to assist in diagnosis and care will require a much better method of data entry than just point and click.

I would like to think that someday HCIT will be viewed like a colleague or consultant on the case that can assist in generating the differential diagnosis or spotting some unique data that requires a second look. I find anchoring bias one of the more difficult habits to break, and an EMR that gently reminded me that I was dismissing a key piece of information would be invaluable.

Dr. Joe,
Wonderful info. And I strongly agree on your reference to Flower's "Five Things" also right on.

As a layman (non-MD) who has worked on all sides of this industry for 35 years to me it's almost simple. Cognitive medicine is based on data and judgement - far more judgement than data. Copgnitive medicine is more art than science (although it does need a solid basis in science) and I do not think that in my children's liftime there will be enough computer power to really address the judgement component adequately. Maybe in my grand childs lifetime with quantum chips and fuzzy logic..but definitely not with today's tools.

Keep up the great posts,
Frank Poggio
The Kelzon Group

Joe
Thanks for highlighting the cognitive issue in your post. Since diagnosis is the first and most important decision made about the patient I am surprised that the issue does not get more attention.

As founder of a company that produces a diagnosis decision support system (Isabel at http://www.isabelhealthcare.com), I have to take issue with you and Frank Poggio about computers being useful here.

I agree that computers currently cannot and probably never will or even should replace a clinician's judgement. Where previous attempts at diagnostic decision support in the 1980s went wrong was that they tried to be experts and better than clinicians.

With diagnosis, the research shows you that the most accurate predictor of diagnostic accuracy is the quality of the differential. If the clinician never thought of a diagnosis in the first place because of a lack of knowledge or inability to synthesize the information then he or she is unlikely to get the diagnosis right. The problem is that the diagnostic process relies far too much on a clinician's memory. This is where computers are really good so, if a computer can come with a list of likely diagnoses for a set of clinician features (what Isabel does), then that is going to go a long way in helping the clinician get to the right diagnosis as fast as possible. What diagnosis decision support can usefully do now is remind the clinician of likely diagnoses when there is doubt and, therefore, support good clinical judgment.

These tools are being used today by leading health systems and can be easily integrated into an EMR.

Regards
Jason Maude
Co Founder Isabel Healthcare

Frank, Thanks for the kind words and sentiment validation. 


I think the framework of Art -vs- Science, perhaps right brain -vs- left brain, and IQ -vs- EQ, each capture something real about how humans actually think and perform.  That's true for non-MDs, MDs, business people, soldiers, sales people, senior executives in government and industry, moms and dads, etc. They play a role in judgement.   And leadership.  You are exactly correct.  This is not computation, nor is it data or information storage and retrieval.

There's always uncertainty in all of our lives about the future and often a significant component of non-availability of todays facts. For example, what in my genome accounts for my cholesterol level and responsiveness to available classes of therapies. No doctor has proposed sequencing my genes.

I love the cognitive reasoning and error work I cited because, as you point out as well, judgement is not a computational thing. It's nice when there are relevant analytics, such as severity, drug dosing, and classification tools. As Drs Graber and Berner and others including economists have pointed out, the utility of employing computational approaches (e.g. diagnostic CDS) is often too low relative to their costs and impact.  (Yes, we all hope to improve that.)

At the end of the day, effective management practices (meaning discipline around structures, processes and outcomes) play a greater role in expressed judgement than do computational and storage/retrieval technologies. Said differently, you cannot fix the highway safety issues solely by creating a perfect airbag. We need to focus on all of the critically important factors. In the case of healthcare, cognitive factors are one of them, and improving that situation is be largely distinct from today's EHR capabilities as the science above (again, Graber and Berner make clear.)

Thanks again, Frank.

Dr. Bormel,
This is a very complex subject and series you've undertaken. From what I've read on many sites, including this one, few bloggers attempt to deal with such subject matter. You're to be commended.

I can see where you're going with the series, and I find such information quite helpful. You and several others who blog here are being very constructive, and readily agree that EMR/EHR systems are not mature but evolving and will continue to do so for some time to come. This brings me to my question.

Where do you stand on studies such as the Stanford research released on Monday wherein the authors conclude that the technology has, to date, had little positive effect on quality of care?

Considering the fact that these systems are not all that far out of the prototype stage, it seems to me such research is of no value because the systems are not yet mature enough to provide sufficient data to validate or invalidate their performance. In my opinion, all such studies do at this point is to cast undue doubt on what amount to large, long-term investments. To draw an analogy, they are much like computer critics in the early 1980s comparing the capabilities of an IBM XT to what visionaries were projecting for the future. I certainly hope Stanford did not receive a federal grant to produce this drivel.

Am I off base thinking that such studies are short sighted and send the wrong message? Worse, when picked-up by the news media for consumption by the general public, system credibility can be severely damaged. I find that disturbing.

Doc Benjamin

The primary reason I believe VisualDx is used and valued is that it meets a well known and persistent need in medicine and that is pattern recognition.  Atlases of all kinds are historically some of the most used and tattered books in medicine, particularly in primary care and emergency medicine.  The statistics show that skin and mucosal complaints make up roughly 20% of all primary care visits, either as a primary or secondary complaint during the visit.   Primary care and emergency clinicians see a lot of skin and mucosal presentations.  It is not just warts and acne, so many infectious, immunologic, metabolic and heritable disorders present with skin findings.  Furthermore, unlike other areas of medicine, most non-dermatologists readily admit they were inadequately trained in the skin exam and dermatology.  So when you address a need, and there is a precedent for many clinicians to seek help as they did with atlases, presenting a quicker solution means you will get used.  

Now consider a patient presenting with chest pain, abdominal pain or other common internal complaints.  For chest pain for instance, most internists, ED physicians and other primary care physicians have a memorized algorithm they employ to work up the patient.  There is simply not a perceived need for many internal problems because doctors often rely on testing and radiography (I’m not saying that CDS would not help reduce error with these complaints) and therefore the clinicians don’t believe they need help. 

But what we do with VisualDx is empower decision with knowledge before testing and radiographs…..that is what visual recognition is all about.  It is much easier to grow adoption with busy physicians if you meet a need. Diagnosis without laboratory or radiographic testing.....  The patients are really impressed!
 
VisualDx is used at over 1,300 hospitals and clinics, now throughout the VA system (since Oct 2010) and over 30 academic centers.  We have a number of medical schools that have embedded VisualDx into teaching (UCLA in particular is being very innovative with VisualDx), and the students are using it on the desktop as well as their IPhones and Itouch devices.  We have the Android version developed and now used in beta.   Mobile access is key to fast paced environments such as the ED or even on walk rounds.  Having the ability to see these world class images and build a differentials from patient findings on the go is very empowering for clinicians, residents and students.
 
The greatest barrier to getting busy clinicians to use diagnostic CDS is that most physicians were trained to look up by disease name and not by patient findings.  So many clinicians do not even realize they can type in a symptom, physical finding, medication or other clue to the differential diagnosis.  So we allow the user to come at a problem from either the presumptive diagnosis or from findings entry

It’s just in our nature to rush to diagnostic hypotheses rather than a slower, more deliberative process with decision support.  Pat Croskerry and Mark Graber and others have published wonderful articles on premature closure, overconfidence and the other cognitive mistakes made by physicians….the problem is that most doctors feel is the other doctor making the mistakes, not them.  Autopsy and other studies suggest that the diagnostic error rate is at least 15%. 

If you talk to patients, friends etc..they almost all will tell you a story about how a doctor at some point got it wrong.   So a great challenge is “how do you know what you don’t know?” this is a generic problem for anyone working in CDS.  Fortunately many clinicians when seeing an unusual visual pattern of the skin, mucosa, hair, or nails want help and know they need help.  There is just something about pattern recognition and visual display that transmits knowledge more efficiently.  That combined with meeting a need, I believe is the reason for our success.
 
Art Papier MD
Chief Medical Information Officer

3445 Winton Place . Suite 240 . Rochester NY 14623
(585) 427-2790 x230 . apapier@logicalimages.com

Art, Interesting, it sounds that VisualDx has broken through the usage problem. The literature does not show widespread use of diagnostic decision support in the past, excellent systems such as the DxPlain, PKC, QMR and Illiad have never achieved widespread use.

How and where is VisualDx used? And why is it used now more than other efforts in diagnostic decision support? What are the barriers and challenges of getting busy physicians to use these systems?

Art,

Thanks for your thoughtful and insightful reply. I added some line breaks and all of the bolding. I hope I did not impact your meaning. Over the years, I've received feedback from readers - make the posts and comments more scan-able. As a result, I occasionally err on over-doing it.

I urge readers to carefully read Art's comment when they're a little less hurried than usual. He shares observations about physician problem solving and decision making that are profound. They are completely consistent with what I've personally seen, as well as experts I've read.

Joe,
 
Thanks for the great blog post and to the participants for such a thoughtful thread.  There are so many points that could be made about diagnostic decision support and the current health IT climate that is difficult not to write a 5 page post!  So I will force myself to focus on the need to distinguish between the widely varying types of clinical decision support.
 
Full disclosure: I am a founder and developer of the diagnostic decision support system VisualDx.  In our work bringing VisualDx to clinicians over the past decade I have seen an amazingly broad spectrum of physician users of health IT,  ranging from incredibly adept “power users” on the one hand, to physicians that are at best skeptical computer users, or at worst physicians who even avoid email. Yet many commentators on health IT want to lump physicians into a broad bin, saying for instance that busy physicians will not use information technology or that information technology does not improve quality.

It’s amazing how I hear physicians characterized as some sort of monolithic block and similarly just as physicians are lumped into a single category so is the diverse field of decision support.  Decision support is not a single technology or solution …decision support is characterized by many different strategies and solutions.  CDS systems are as varied as reminders, alerts, medication decision support, laboratory guidance, and diagnostic decision support to name a few areas. 

The Romano and Stafford article was titled Electronic Health Records and Clinical Decision Support Systems.  While they were studying specific types of CDS interventions, they titled the article generically.   As a developer of a diagnostic decision support system, this over-reaching title is painful.  The title should have included the specific type of CDS studied and is a disservice to those working in other areas of decision support. 
 
In the area of cognitive support the facts speak for themselves; twenty years ago or less, medical students read books, tried to memorize as much as they could, and then moved into practice (rarely updating the textbooks they purchased during medical school or residency).  Now students, residents and physicians routinely “pull” information from tools such as UpToDate and VisualDx.

Now information can be at their fingertips on the desktop or mobile devices.  Why would anyone want to debate what the marketplace has decided?  If textbooks are so great why is the print medical publishing industry declining?  CDS is evolving swiftly and powerfully.  

Another deficiency of the current US health IT direction and discussion is that health IT has become synonymous with the electronic medical record.  This is unfortunate.  There is so much more to health IT than the medical record.   The slow penetration of electronic records and the wisely cautious and slow speed of integrating decision support into the record does not mean that diagnostic decision support is not being used.  It is, and it is improving every day!  

So we repeatedly hear that physicians will not use decision support tools.  This is simply not true.  VisualDx (www.visualdx.com) is licensed into 1300 hospitals and clinics including the entire VA health system.  We have millions of images viewed for education and diagnostic purposes each year (and growing)……..We have had physicians tell us of diagnoses such as secondary syphilis and acute meningococcemia that were made with the assistance of VisualDx. 

As a profession we should showcase the clinicians using all the technologies available to improve the care of their patients and we should strive to highlight cognitive error and systems to improve diagnosis.  It’s a shame that “meaningful use” is  solely focused on the medical record, and not these powerful other systems that can be used within the context of the record or simply pulled from the “cloud” on the desktop or mobile device.   There needs to be more attention on mobile computing, diagnostic decision support, telemedicine, and a host of other innovations that currently are not receiving the attention they deserve.
 
Best Art
 
Art Papier MD
Chief Medical Information Officer

3445 Winton Place . Suite 240 . Rochester NY 14623
(585) 427-2790 x230 . apapier@logicalimages.com

Thanks Doctor G both for the kind words, as well as the sobering assessment.  There is a dimension to the diagnostic uncertainty and practice environment that is critically important and deeply experiential.  You bring that sharp dimension to every comment.  Thank you. 
 
The human persona, the physician perspective on the human side of the keyboard is often lost in the retrospective narratives that most of our EMRs are designed and modeled to support.  I'm sure there are several biases captured in that observation!  For example, when we see a patient for chest pain, before you know if it's a pulmonary embolism, heart attack, dissection, or costochondritis, today's EMR are completely naive regarding the issues and urgency issues.  Once the results of the spiral CT come back, it's rare that an EMR will change it's flowsheets, orderset options and relevant documentation recommendations.  Many are capable of more nuanced behavior, but, as you suggest, that's not how they're rolled out to end-users today.  
 
Readers interested in the topic of how computers might specifically inform us to reduce some cognitive biases should read "The Butler Model" (here).    That post is from exactly one year ago and received some great comments, both publicly and privately from Dr Graber.  It was also one of the more fun posts to write, which is probably obvious.
 

Pages