A voice interface for electronic health records

Electronic health records (EHRs), mandated in the US by the Affordable Care Act, have become an integral component of our medical care system. Ton Ngo and Yi-Hong Wang explain how they are using TensorFlow.js to build a voice interface for a web-based electronic health record system. This project is a part of the Call for Code Initiative announced in May 2018, which calls for using technology to help disaster relief.

The goal of this project is to leverage the advantages of EHRs when a major natural disaster hits a large population in remote area. In these situations, survivors require medical care while dealing with chaotic, primitive conditions. To help nurses and doctors be more effective in providing care, Ton and Yi-Hong envision deploying a local EHR system in the field. However, one of the challenges in operating in this situation is entering patient data into the EHR. Laptops may not be convenient in the field and are likely not available, but smart phones will likely be ubiquitous. Since these devices have microphones and can run a browser, a voice interface to the EHR system should make it possible to enable easy data entry.

To implement this voice interface, TensorFlow.js in a web-based EHR is used to load a speech-to-text model and manage the interaction between the user and the app. The JS interface is specifically designed to fit an appropriate workflow for the medical care providers in the field. The project leverages publicly available deep learning models and datasets, such as the Google tensor2tensor code, the Mozilla Deep Voices project, and the Figure-Eight medical transcription dataset. The model initially will support English, with the possibility for other languages later. Ton and Yi-Hong describe the design, progress, and the challenges they encountered.