Live From HIMSS 2011 — Rookie Mistakes, the Rule of 888, and Usability | Joe Bormel, M.D. | Healthcare Blogs Skip to content Skip to navigation

Live From HIMSS 2011 — Rookie Mistakes, the Rule of 888, and Usability

February 22, 2011
| Reprints
Important takeaways from the HIMSS11 Physicians' IT Symposium

The Physicians' IT Symposium keynote was provided by Farzad Mostashari, MD, ScM from the Office of National Coordinator for Health Information Technology. Dr. Mostashari, in talking about his early experience with EHR projects, referred to “rookie mistakes,” which were common errors of misplaced exuberance with one's first major clinical transformation project.

There were no slides and I don't have access to the transcript yet, but I heard that one of the most common errors was trying to do too much, including using more features than necessary, and “continually insisting on adding new features/functionality” as go-live requirements. This set a wonderful tone, and several subsequent speakers contextualized some of what they had prepared to share what they learned from their rookie mistakes. There was agreement, at this symposium and others such as the CDS-based sessions, that the biggest recurrent rookie mistake was putting in EHRs as though they were technology (aka IT) projects.

Dr. Mostashari urged “simplicating” (the opposite of complicating, and a new word for me) as an important antidote to approach health care transformation and related technology-enabled modernization projects. He urged us all to remember “The Aim Makes The System.” You should always start planning sessions, awareness, acceptance and commitment phases of change management with “tell me what you want to achieve,” or a related “norming” discussion. This is, of course, great advice, because omission, i.e. starting meetings without clear framing on goals (or aims), is a common rookie mistake for people in all industries, not just HCIT.

Dr. Bill Bria, AMDIS representative and co-founder, pointed out several hours later that, as clinical leaders working in the space that includes technology-enabled process, workflow and quality improvement, we still have what seems like the majority of physicians seeing EHRs as technology transfer initiatives, not as essential tools for the practice of medicine. Arguably tools far more essential than the stethoscope. It's much easier to argue that technology is unusable than it is to argue it is essential to have adequate information about a patient and be able to reliably diagnose and treat a patient in a timely, and again, reliable manner. His message was that CMIOs and CMOs don’t routinely get out and advocate what the modern practice of medicine is, what’s needed and possible, framing clinical knowledge management and execution, not the EHR alone, as central.

Here are two important takeaways:

1. The Rule of 888 or less. This came up in a variety of contexts from easily a dozen different presenters. In simplicating your project, whenever it appears there are thousands of items to manage, work aggressively to get that number below 888. For example, one team studying the right number of order sets to roll out determined that just under three hundred ensured that the build, maintenance, revision and updating work was kept to a manageable number. Several of the smartest informatics experts, working independently, have advocated that creating data liquidity by reducing and agreeing on a manageable number of problems, allergies, medications, quality measures, lab findings, etc., can and should be published after rapid harmonization by NQF. Why NQF? They are the neutral conveners who are reconciling the quality measures to measures that will be capable of promoting safe, effective and stable measurement, comparison, and ranking of performance. This derives from active work on eMeasures underway from ONC sanctioned work being done by MITRE.




The key to software usability is consistency. Users can be productive and "happy" with limited green-screen character-based software if their on-screen actions lead to predictable and consistent behavior. In other words, no surprises, even when new features are added later. Someone with an eye for detail, with authority, has to enforce that consistency.

But usability is not enough anymore. Software should be barely noticed, potential actions should be anticipated, explicit and obvious, and software should be used productively with less than 15 minutes of training. I hate to use an over-used term, but these features describe "intuitive" software. Once consistency is a given, misplaced buttons, error-prone actions, and super-secret actions should be vanquished. Someone with their eyes wide open, with authority, has to enforce that software is a joy and easy to use.

To achieve both usability and intuitiveness, repeated observational testing with fresh, naive, first-time users is required. Anything else is a shortcut.

P.S. If software vendors do not anticipate and deliver the desired content, appropriate to a clinician's workflow, then never mind about usability ;)

It's also important to learn from observing the current world.  Usable and intuitive are not necessarily the same.  Many green screen systems from the past  (command line-oriented that worked on terminals without "windows, icons, mouse, and [mouse] pointers") were highly usable and generally screamingly fast.  They were rarely intuitive.

Thanks again for your comments. It's absolutely clear that you have a deep understanding of software development and usability, both from your writing and from my knowledge of your great work.

Congratulations! You are the first to call out the ironic juxtaposition of Dr Mostashari's simplicating with Dr Friedman's introduction of it's opposite!

Regarding Simplicate:

To dynamically match the anticipation and focus of a clinician standing in front of a patient by processing imbedded knowledge systems with hundreds of rules, and multiple linked data repositories and domain jumps may indeed tax the performance of even the "big iron" and server farms available today. But if this processing takes more than 3-4 seconds, that kind of anticipatory feature is a non-starter and a source of aggravation (Steve Jobs wouldn't let it out the door). Of course, there are tricks to hide such performance roadblocks, like anticipatory background processing, late-night batch processing, canned knowledge-based templates, and gradual reveals of information as it arrives. The most obvious approach to this state is to "simplicate" and lower expectations from a dream state that cannot currently exist to a very usable "good enough" state where the key part of the job gets done and the user can move on. Unfortunately, most EMR vendors do not choose to sell "good enough", and those choosing systems may have watched too much Star Trek and reflexively dismiss "the good" (and useful).

Steve Jobs also may not have ordered usability testing for the physical aspects of the iPhone, but scores of engineers, graphic designers, and usability experts developed the iOS SDK (widgets) and the accompanying Human Interface Guidelines (HIG) which almost defines usable software products for the iPhone/iPad. Microsoft has published HIGs since the earliest versions of Windows which were largely ignored by many of those developing multi-million dollar EMRs with each of their engineering team leaders deciding the level of adherence to those guidelines.

"Fresh user" testing relates primarily to avoid the expense of training (which remains a major source of revenue for EMR vendors) and making use of the software explicit to all (i.e. not aimed at power-users - and software engineers - who know all about right-mouse clicks, drag and drop, etc.). Driving down a street even once and learning where the potholes are does not help the next unwary driver down that same street. The goal is no potholes. For my two cents, the most-widely tested, most understood user-interface metaphor is from the web browser, i.e. click/tap on something that is underlined or changes colors. No one has to train anyone to shop Amazon.

Simplicating. (Word was not found in Merriam-Webster, Encarta or Cambridge dictionaries on-line.  Nor was it found in the NLM’s MedLinePlus, but that uses Merriam-Webster, and hey, who’s counting anyway?)
Synonym: I’m from Washington and I’m here to help you.
Tell me how talking the talk and walking the walk are in synch.

  • Will it simplicate clinicians’ lives if they are made responsible for providing patient education around good privacy practices on their EHRs and documenting whether or not patients have internet access
  • Will it simplicate clinical decision making to require clinicians to read through reams of ‘information’ that patients will be able to enter directly into their EHR?
  • With the underlying policy purpose of MU being to adopt and use EHRs for good patient care, does it simplicate adoption by requiring physicians to support automated patient reminder systems and patient portal systems?
  • Will it simplicate outcomes measurement to report 114 e-measures?
  • Will it simplicate everyone’s life by having a Final Rule in June of 2012 and a mandatory go-live date of October 1?

I would suggest that, while each of those except the timeline has intrinsic merit, they are all well outside the 888 rule.  Why is it then that they persist inside the beltway?

Thanks Don.

You have brought up several critical aspects of usability that have magically been left out of many of the public and private meetings on usability. Specifically:

1) Consistency.

2) Given human nature, the need for consistency police, both at vendors and provider organizations. Custom reports, custom apps, custom workflows, and use of obscure "standard" features that no one is familiar with are usability killers. Is every call to the help desk an indicator of a design flaw? A hazard? Too many are.

3) Usability, really good usability, often implies anticipation. Historically, there are significant limits to this in practice because of the performance issues of transaction systems (OLTP). That's not a simple cop out. For example, anticipating pre-screening ordering options to DDI, DAI, duplicate checking, P&T substitution, dose-range, etc is pretty much never done. Certainly not exhaustively. The result? The computer can take a minutes of a users time, down a path that will, predictably lead to disappointment and anti-productivity.

4) Response-time (screen or page flip completion) as a usability metric.

5) The repeated, fresh user test is one I didn't expect in your comment. I have read that Steve Jobs did zero usability testing with consumers on the iPhone. He was quoted as saying something like "it's not reasonable to expect them to clearly and explicitly know what they want." It's our job is to produce the best user experience. A great example is the decision around flash. It's great for compatibility; it's a home-wrecker for performance, battery life and the requisite road map. It may be more complicated but backward compatibility and "persisting" usability don't always mix.

6) Clinical task complexity has been absent in most of the demonstrations of "great" or exemplar usability this past week at HIMSS. Observing the number of clicks and time it takes an "expert user" to perform an ARRA/NIST certification test step (e.g. add a medication allergy) in isolation of a workflow (such as an ED triage with medication reconciliation), while interesting, is irrelevant.

Over a decade ago, Dr Jim Walker, now CHIO at Geisinger introduced me to the usability and workflow challenge of a doctor receiving an arbitrary INR (INR is a lab result used in anticoagulation.) Systems with poor clinical usability will show that INR in a simple, clean screen, by itself. An "enhanced" system may show it in a spreadsheet with prior INRs (the trend, possibly graphically depicted as well, by default). A further enhancement might show the patient's prescribed or actually administered Coumadin dose.

How does a truly clinically usable system address this task? The true task, while still relatively simple is actually compound, when built out by highly competent and usable internal developers and vendors. As described at length elsewhere in my blog, to be usable requires concurrent:

a) guided results review, including the problem list, highlighting the reason and goals of anti-coagulation related to the INR

b) guided relevant documentation review and capture

c) guided communication with the patient (e.g. show their phone number, or networked PHR communication functions),

d) guided ordering

e) integration with a care coordination system so that goals, sick-day rules, and guidelines are integrated in the dialogue passively (as opposed to requiring a user to note a color change or drill down to an alert)

f) in the spirit of high-reliability organizational thinking, i.e. a pre-occupation with potential failure and early containment, executable knowledge; so
     i. vitals and labs that would portend side-effects (evidence of occult blood loss, deteriorating vital signs, etc)
     ii. orders or protocols regarding the antidote, Vitamin K, if the INR is significantly elevated or signs of bleeding are detected

g) and the ability, as a byproduct of using this usable workflow, the documentation regarding what was seen, considered, intended and done are all captured with one click or gesture. Wouldn't it be more usable if the same paradigm applied to Medication Reconciliation, documenting what was done, that it was done completely, and why it was done?

This is simply an elaboration of what you, Don, are describing with anticipation of what the job requires. That's much closer to true clinical usability, and far away from single-function usability testing that is highly feared to be the path we are on. Especially when no MD or DO informaticians are responsible or co-responsible for the usability testing design decisions. Being consulted on a decision is an order of magnitude less influential than owning it. It's one notch better than being informed of a decision, after it has been irrevocably made.

Am I understanding your points?

Readers interested in this topic are encouraged to review Alan Cooper's book, The Inmates Are Running the Asylum: Why High Tech Products Drive Us Crazy and How to Restore the Sanity .  This was recommended to me by Don over a decade ago.  It continues to inform my understanding of usability and it's opposite.  So does trying to support my parents on using their computers ;)

The Route 7 Story

On my first day of HIMSS, as depicted by the opening photo, the pedestrian walkway led straight into the body of a parked bus. Attendees had to walk out of the safe, marked walkway and around the back of the bus. The bus was, of course, running and producing typical waste gas. This was all an unintended consequence of Route Seven stop placement.

To the credit of some one or some ones, within a day, they were parking the Route 7 bus such that it did not block the pedestrian walkway.

Mildly flawed design > Observant, caring person > simple, timely fix. Kudos to HIMSS for responding in real time to a minor design flaw.