Hardware, Software & the Internet of Things
June 23–25, 2015 • San Francisco, CA

Designing for Dialog: Speech and Transmodal Design for the Internet of Things

Tony Sheeder (Nuance Communications, Inc.)
5:25pm–6:05pm Thursday, 06/25/2015
Location: Southside Theater (Bldg D)

Prerequisite Knowledge

No specific skills will be required. A general/basic understanding of connected/smart devices and a general/basic understanding of speech technology will be helpful for attendees.


The promise of the Internet of Things (IoT) can be found in that final word – “Things.” We have more connected devices than ever before, and not just TVs, watches, cars, and thermostats, but more traditional appliances like refrigerators and washers, and even things best categorized as, well, things – the hubs on our walls, the screen-less speakers, and countless other objects that probably haven’t even been thought of yet.

Historically, interactions with devices have been specific to the individual device: laptops had mice and keyboards; smartphones had touchscreens; TVs had remotes; etc. But many of the emerging IoT devices don’t have specific, established interaction methods. Will that connected refrigerator have a keyboard, a touchpad, a smartphone application, or something else entirely? Extending this question to different brands, operating systems, and device ecosystems, it becomes clear that one of the challenges of the Internet of Things is creating ways of interacting with these devices that are consistent and intuitive.

We can draw inspiration from our most natural form of interaction: speech. Speech can be a unifying interaction modality for the Internet of Things – one that transcends or complements screen shapes, screen sizes, menus, buttons, and all of the other input methods that will necessarily vary from device to device. If the Internet of Things is supposed to keep us connected more simply and conveniently, our interaction experience should also be simple, convenient, and consistent. Speech can provide that that unifying interaction model.

Have you ever wondered how speech interactions actually get designed and implemented, and how they work with different devices? “Designing for Dialog: Speech and Transmodal Design for the Internet of Things” will address:

• The challenges of interacting with an expanding range of connected devices
• Speech as a consistent modality, promoting common experiences as people move from device to device; remaining specialized but accessible, intuitive, and effective
• An overview of speech experience design, including considerations for human conversational norms, boundaries, and transmodality
•Speech experience design considerations across different devices – some boasting small screens, large screens, no screens, etc.
•Speech as a standalone modality and a complement to pre-existing modalities, such as touch and gesture.

Photo of Tony Sheeder

Tony Sheeder

Nuance Communications, Inc.

In his role as a Senior Manager of UX Design, Tony directs an interdisciplinary group of visual, voice, and interaction designers focused primarily on automotive applications – creating the next-generation in-car experience – as well as on the design of innovative transmodal applications – incorporating speech, touch, text, and gesture – for interactive television, wearable, mobile, gaming, and other in-home platforms. With over 15 years of experience designing for speech, Tony has monitored new technologies, helped articulate and communicate Nuance’s creative vision and product strategy, created reference designs for emerging products, and overseen the development of Nuance Voice Ads, a product that allows marketers to create rich, immersive conversational experiences in which consumers literally talk to brands