Presented By O’Reilly and Intel AI
Put AI to work
Sep 4-5, 2018: Training
Sep 5-7, 2018: Tutorials & Conference
San Francisco, CA

Reverse engineering your AI prototype and the road to reproducibility

Brian Dalessandro (Zocdoc), Chris Smith (Zocdoc)
11:05am-11:45am Friday, September 7, 2018
Implementing AI
Location: Yosemite BC Level: Intermediate
Secondary topics:  Deep Learning models, Platforms and infrastructure

Who is this presentation for?

  • Data scientists, engineers, and technical managers

Prerequisite knowledge

  • Familiarity with AI

What you'll learn

  • Explore Zocdoc and learn how the company reverse-engineered an optimized prototype and converted it to a fully reproducible AI pipeline


Deep learning has gotten considerably easier in the past few years, with open source software, cloud-based infrastructure, and pretrained networks enabling fast and accurate prototyping of AI applications. But what happens when your AI task quickly becomes nontrivial? Brian Dalessandro and Chris Smith discuss Zocdoc’s road to a production-ready, reproducible AI system for insurance card classification and OCR.

Zocdoc is an app that allows patients to find and book doctors online. In additional to facilitating online medical scheduling, Zocdoc makes it easy for patients to check in online and precomplete their medical paperwork. Its insurance card scan product uses an uploaded insurance card to infer the patient’s insurance plan and verify coverage for their medical appointments—a computer vision task that is a natural fit for AI applications.

Early experiments revealed that off-the-shelf models would not drive the needed level of accuracy, especially for the task of localizing key fields and extracting text from images. After months of experimentation and generally “dark art” fiddling, Zocdoc reached its end-to-end accuracy goals using a multitask architecture that combined CNNs and RNNs with attention mechanisms. The next challenge was to reverse engineer its own work and build a system that was fully reproducible and easily extendable to enable automated and iterative improvements.

Brian and Chris offer a general overview of Zocdoc’s really cool, great-for-patients, state-of-the-art classification and OCR system, focusing on general lessons and tips for promoting your AI system from the lab to a fully automated, production-level, self-learning AI pipeline.

Photo of Brian Dalessandro

Brian Dalessandro


Brian Dalessandro is the head of data science at Zocdoc, an online doctor marketplace and booking tool, and an adjunct professor for the NYU Center for Data Science graduate program. Previously, Brian was vice president of data science at online advertising firm Dstillery. A veteran data scientist and leader with over 15 years of experience developing machine learning-driven practices and products, Brian holds several patents and has published dozens of peer-reviewed articles on the subjects of causal inference, large-scale machine learning, and data science ethics. Brian is also the drummer for the critically acclaimed indie rock band Coastgaard.

Photo of Chris Smith

Chris Smith


Chris Smith is a senior principal software engineer at online doctor marketplace and booking startup Zocdoc, where he is developing deep learning models to help Zocdoc’s patients understand the complex world of medical insurance. Through machine learning, Chris is helping to bring data-driven products to an industry traditionally resistant to change. Over his career at Zocdoc, he’s built numerous patient-focused products, rebuilt the company’s CI/CD systems, helped migrate Zocdoc from the data center to AWS, and set up the first version of its microservice infrastructure.