Put AI to work
June 26-27, 2017: Training
June 27-29, 2017: Tutorials & Conference
New York, NY

Human-assisted AI at B12: 10 lessons in giving humans superpowers

Adam Marcus (B12)
11:05am11:45am Thursday, June 29, 2017
Implementing AI
Location: Sutton Center/North Level: Intermediate
Secondary topics:  Machine Learning
Average rating: *****
(5.00, 2 ratings)

Prerequisite Knowledge

  • A working understanding of how model building works and the differences between rules-based and statistical models (no mathematical understanding of these topics required)

What you'll learn

  • Learn ideas for utilizing human-assisted AI in your own domain, as well as pointers to systems and best practices to avoid having to do everything from scratch


AI is positioned as a zero-sum game: either a human will complete a task, or a machine will do it instead. While the field is advancing quickly, it’s hard to see an AI replacing multifaceted and deeply creative jobs like designer or data scientist in the near future. These fields are actively being changed, however, by a different paradigm known as human-assisted AI.

Human-assisted AI elevates human capital instead of replacing it. In human-assisted AI, machines eliminate the most rote and repetitive aspects of a job, allowing experts to focus on the uniquely challenging aspects of their jobs with fewer interruptions. Adam Marcus offers an overview of human-assisted AI and demonstrates how it is already changing creative (and fundamentally human) fields like design.

Topics include:

  • How B12 created a human-assisted AI for the deeply creative field of design, allowing its web designers to hone their craft with the help of automation to collect content about clients and build initial versions of complete websites.
  • What infrastructure is helpful in building this human-assisted AI future. B12 has built and open sourced Orchestra, a system that helps coordinate teams of experts who are working on challenging workflows. Orchestra pairs experts with automation, custom user interfaces, and tools to make them more effective at their jobs. Adam touches on the research behind the system, including the Flash Teams and Flash Organizations research at Stanford.
  • The challenges and opportunities for utilizing human-assisted AI in your own creative and analytical tasks. While systems and software make it possible, there are several best practices in recruiting, training, and elevating the experts that power these sociotechnical systems.
  • How to integrate different forms of intelligence into human-assisted AIs, ranging from general human intelligence to deep expertise.
  • Some patterns for training progressively more complex models that learn from humans in the loop as your system collects more and more data.
Photo of Adam Marcus

Adam Marcus


Adam Marcus is a cofounder and CTO of B12, a company building a better future of creative and analytical work, starting with design. With Orchestra, its open source project management system for experts and machines, B12 automatically generates websites for clients (algorithmic design) and then recruits wonderful designers and art directors to fill in the details from the algorithmically generated starting points. (This summer, B12 announced the close of a $12.4M Series A funding round.) Previously, Adam was director of data at Locu, a startup that was acquired by GoDaddy. He has written widely on crowdsourcing and data management and processing, including coauthoring a book, Crowdsourced Data Management: Industry and Academic Perspectives. He is a recipient of the NSF and NDSEG fellowships and has worked at ITA, Google, IBM, and FactSet. Adam holds a PhD in computer science from MIT, where he researched database systems and human computation. In his free time, he builds course content to get people excited about data and programming.