Every vendor involved in ARRA certification, inpatient and eligible ambulatory care provider, has rolled its newly certified code to a beta client or two, on the path to demonstrate Stage 1 Meaningful Use. This will be the first relevant experience for implementation teams to bring Stage 1 functionality live across the install bases.
What is a beta test? Definition
Because of the incentive revenue implications for the provider organizations, and the threat of having their attestation contested by the government, and because of the complexity of dozens of measures (Core, Menu, and requisite capability), we have never before seen betas of this complexity in HCIT. And just as we begin to tie a bow around our Stage 1 accomplishments, an uncertain Stage 2 looms on the electronic horizon.
Stage 2 Meaningful Use is shaping up to be more complicated with a broad spectrum of escalating requirements. Already many on both sides of the vendor-provider partnership are pushing back against what they’ve determined is an overly challenging set of requirements with an equally unrealistic deadline. Stage 2 certification will be a difficult and expensive process, but with commensurate improvements for more efficient, safer and better care delivery.
Pilot testing (see definitions of alpha and beta stages above) for both Stage 1 and 2 requirements are, of course, extremely important. There have been many past pilots from many very competent IT companies that ended in disaster even though the controlled testing results were positive. In this blog, I hope to help those of you developing, managing, participating in, and interpreting pilots to avoid having to deal with a disaster by implementing a crisis management plan focused on not rolling out a pilot program designed solely to succeed.
A short time ago, I ran across a section of an article on pilot roll outs. The article was titled, “ Strategies for Learning from Failure ,” by Amy C. Edmondson. Read the whole article. It’s a treat, and is typical of her other research. There is a companion video interview with the author here .
The article closes with an interesting twist on pilot testing that’s worth your attention:
Designing Successful Failures
Perhaps unsurprisingly, pilot projects are usually designed to succeed rather than to produce intelligent failures—those that generate valuable information. To know if you’ve designed a genuinely useful pilot, consider whether your managers can answer yes to the following questions:
• Is the pilot being tested under typical circumstances (rather than optimal conditions)?
• Do the employees, customers, and resources represent the firm’s real operating environment?
• Is the goal of the pilot to learn as much as possible (rather than to demonstrate the value of the proposed offering)?
• Is the goal of learning well understood by all employees and managers?
• Is it clear that compensation and performance reviews are not based on a successful outcome for the pilot?
• Were explicit changes made as a result of the pilot test?