Design the Future
January 19–20, 2016: Training
January 20–22, 2016: Conference
San Francisco, CA

Direct manipulation is broken: Why the IoT asks consumers to think like programmers and the UX challenges this creates

Claire Rowland (Independent)
3:45pm Thursday, 01/21/2016
The new fundamentals
Location: Southside Theater
Average rating: ****.
(4.50, 2 ratings)
Slides:   external link

In direct manipulation—the basis of most examples of successful consumer user experience design in the last 30 years—users act on UI objects in front of them instead of learning a system of abstract representation. Direct manipulation user interfaces enabled personal computers to become mass market. But the Internet of Things (IoT) breaks direct manipulation. IoT systems allow remote control, automation, and the ability to coordinate multiple devices. When done right, these can be powerful features.

But this causes significant UX challenges, particularly for consumers. User actions may produce consequences at future times, and in different places. Users must interact with abstract representations of devices, actions, and notation systems to coordinate devices (like IFTTT or smart rules). They may not get immediate or complete feedback on the consequences of their actions. Even if they can undo problematic actions, they may not know they need to.

Conceptually, this is much more like software engineering, where users must anticipate their own future needs and plan a solution, configure the system to meet these needs using an abstract representation, and deal with unforeseen conditions that may make the system behave in unexpected ways, all while considering future conditions that might cause unexpected effects or conflicts.

Empowering the widest possible audience to “program” their world is a noble aim, but for many users, this poses cognitive challenges and demands a heavy investment of time and attention. Claire Rowland explores how using the IoT is like programming and why that proves a challenge to many consumers. In the process, she addresses key considerations for designers, including to what extent it is feasible to offer consumers easier, quicker routes to do programming-like things, how we might make these easier to grasp, and when it is better for the system to be autonomous. . .without the user losing control.

Photo of Claire Rowland

Claire Rowland


Claire Rowland is an independent UX design, research, and product strategy consultant working on internet of things products and services for mainstream consumers. She is the lead author of Designing Connected Products: UX for the Consumer Internet of Things, published by O’Reilly. Claire has a particular interest in the use of technology in mundane, everyday activities and has worked in UX design and research for mobile, multiplatform, and web services since 1997. Previously, she worked on energy management and home automation services as the service design manager for AlertMe, a connected home platform provider, and was head of research for the London studio of design consultancy Fjord, where she led Fjord’s involvement in the Smarcos EU consortium researching the interusability of interconnected embedded devices and services.