The digital hearth: Speech-to-text for civics

Wes Chow (Cortico at MIT Media Lab)

2:15pm–3:05pm Tuesday, February 25, 2020

Location: Nassau

Application architecture, Key skills

Secondary topics: Case Study

Who is this presentation for?

Engineers

Level

Beginner

Description

Cortico and the Social Machines Group at the MIT Media Lab are building a network of hyperlocal conversation centers in order to raise unknown and underrepresented issues into public discourse. To do so, they’ve built the digital hearth, a group-conversation recording device deployed into communities to capture speech and ideas.

Wes Chow describes the design and technical capabilities of the digital hearth, which operates disconnected from the internet but periodically syncs its data with Cortico’s servers, downstream speech-to-text, and natural language processing systems. You’ll get an overview of the hardware configuration (a custom eight-channel mic that interfaces with an embedded Raspberry Pi) and the in-device Raspbian-based software stack that allows for offline operation and remote debugging. You’ll also see how features of the hardware implementation affect Cortico’s automated speech recognizer and speaker identification systems.

What you'll learn

Understand how Cortico's digital hearth is a different kind of Amazon Echo—one that's optimized for multiple continuous speakers, typically 1.5-hour-long conversations
Learn how to approach a highly experimental design with continuous development in the wild

Wes Chow

Cortico at MIT Media Lab

Wes Chow is the director of engineering in advanced analytics at Cortico and a researcher at the MIT Media Lab. He’s an advisor to Chartbeat, where he was CTO for more time than healthy. Previously, he built early infrastructure for high-frequency trading shops and led the team that created Songza Radio, which was acquired and merged into Google Music.