At Northwell Health, IT Leaders are Revamping the EHR with AI, NLP and Voice Tools | Healthcare Informatics Magazine | Health IT | Information Technology Skip to content Skip to navigation

At Northwell Health, IT Leaders are Revamping the EHR with AI, NLP and Voice Tools

July 5, 2018
by Heather Landi
| Reprints
Click To View Gallery

For physicians and clinicians, electronic health record (EHR) usability and the time spent box-checking and on data entry are oft-cited sources of frustration and stress. Numerous studies and surveys indicate widespread dissatisfaction among physicians with the time spent on EHR documentation and the impact on patient interactions.

A study published in the Annals of Family Medicine last fall found that primary care physicians spend nearly two hours on EHR tasks per hour of direct patient care. Another time and motion study published in the Annals of Internal Medicine in October 2016 found that, outside office hours, physicians spend another one to two hours of personal time each night doing additional computer and other clerical work. What’s more, a Mayo Clinic study linked EHRs with physician burnout.

Some of the biggest names in technology are working to tackle this problem, as well as innovative teams at many large health care systems, with a focus on developing “smart” EHRs using artificial intelligence and voice recognition technology. As recently reported by CNBC, Google is exploring ways to use AI and voice recognition to improve patients’ visits to the doctor. CNBC reported that four internal job openings at Google describe building the "next gen clinical visit experience" and using audio and touch technologies to improve the accuracy and availability of care.

The project falls under the healthcare group on Google Brain, part of its Google AI division, and is sometimes referred to internally as "Medical Brain,” according to CNBC. “The project would likely take advantage of the complex voice technologies Google already uses in its Home, Assistant, and Translate products,” CNBC reporters Jillian D’Onfro and Christina Farr reported.

Last fall, Stanford Medicine and Google Research announced a pilot project to study the use of a digital-scribe to replace a human scribe in order to save the physician time on data entry and improve physician-patient interaction. The digital-scribe system uses speech recognition technology and machine learning tools to automatically enter the information from the office visit into an EHR system.

Earlier this year, Microsoft announced a collaboration with the University of Pittsburgh Medical Center (UPMC) on its Intelligent Scribe platform, a virtual AI assistant that “listens in” during a doctor’s visit and takes notes. The application analyzes a doctor’s conversation with a patient and then makes suggestions in the patient’s electronic health record.

Back in April, Vanderbilt University Medical Center unveiled a voice assistant for VUMC clinicians to interact with the hospital’s EHR system, developed by VUMC’s biomedical informatics and health information technology innovations division. Using natural language processing (NLP) and AI, the EHR Voice Assistant is designed to understand and fulfill verbal requests from clinicians and other hospital staff.

Additionally, startup companies like SayKara and Notable are also developing solutions in this space. SayKara developed an AI-powered scribe for doctors and Notable developed what it calls “voice powered healthcare” using AI to automate and structure patient-physician interaction.

At Northwell Health, a New York City-based health system with 23 hospitals, clinical and IT leaders collaborated on an innovation project, called EMRbot, that allows physicians and nurses to “talk” with a patient’s medical record. The technology enables users to interact with EHRs using voice, natural language text messages and an adaptive user interface using chatbots, according to the project’s leaders, Vishwanath Anantraman, M.D., Northwell’s chief innovation architect, and Michael Oppenheim, M.D., the health system’s chief medical information officer.

Northwell Health’s IT department plans to begin pilot testing the technology at several hospitals by the end of the summer.

The EMRbot was developed as part of Northwell Health’s third Innovation Challenge, which rewards innovative employee-led projects. The team behind the EMRbot project and another winning project team each received $500,000 to bring their concepts to market.

“EMRbot will completely revolutionize how clinicians interact with patient data and will restore the human `face-to-face’ interaction that EHRs have slowly eroded,” Dr. Anantraman said in statement when the Innovation Challenge winners were announced back in May.

In developing the EMRbot technology, the project team sought to solve an ongoing challenge in healthcare: physician and patient dissatisfaction with how much time doctors spend with computers during the patient visit. “Computers have become very disruptive to the workflow that physicians and nurses want to have, which is at the patient bedside or talking with the patient, more face-to-face interaction. Physicians and patients want interactions that are more natural,” Anantraman says.


Get the latest information on Health IT and attend other valuable sessions at this two-day Summit providing healthcare leaders with educational content, insightful debate and dialogue on the future of healthcare and technology.

Learn More