Lessons from building Facebook's visual cortex

Description

Facebook is a great platform for people to share what they care about, which generally tends to be visual in the form of photos, videos, 3-D photos, or stories. This rich visual content, combined with a social graph, can create a rich semantic graph if you can understand the underlying subject of the media. This requires developing deep networks that can understand people, objects, and the interaction between them. Facebook deploys hundreds of such models, capable of doing simple tasks—like fine-grained classification—to compute-intensive ones—like segmentation of a particular special class.

Training and then running these models at Facebook’s scale, with the final outputs (from hashes to embeddings to concept maps) being consumed by hundreds of downstream applications, brings up a new set of challenges. Roshan Sumbaly explains why, surprisingly, the answer to these problems lies in various well-studied and practiced software engineering best practices—starting from backwards compatibility to continuous integration and deployment. Roshan connects the dots between the software engineering and ML development worlds and gives examples of what worked and what didn’t. There are a lot of lessons that should help any ML practitioner think through the various building blocks while building a production-level vision platform.