Put AI to work
June 26-27, 2017: Training
June 27-29, 2017: Tutorials & Conference
New York, NY

Deep shopping bots: Building machines that think and sell like humans

Rupert Steffner (WUNDER)
4:50pm5:30pm Thursday, June 29, 2017
Interacting with AI
Location: Murray Hill E/W Level: Intermediate
Secondary topics:  Machine Learning, Retail and e-commerce, User interface and experience

Prerequisite Knowledge

  • Basic familiarity with AI, deep reinforcement learning, and model thinking

What you'll learn

  • Understand the cornerstones of real-world AI applications that interact with customers
  • Explore deep shopping bots


In the coming years, we will see various kinds of machines thinking and learning like humans. Rupert Steffner offers an overview of the cornerstones of building smart interactive bots for selling and explains why they will do a far better job compared to online selling happening today. In an attention economy, shopping bots that help overcome choice or information overload have to become more sophisticated. Rupert shares examples of deep shopping bots based on true personal relevance—they try to read customers’ minds, know all the products in the market, are playful and conversational, and act and learn in real time.

Topics include:

  • Layering models for a more generalized AI approach: Specific models often result from trying to put everything into one model. Instead, two models are combined to achieve a more generic cognitive AI approach. The first is a high-level model that goes for the true personal relevance shopping context by matching consumer personalities directly against product personalities. The second goes deep (deep shopping) by defining the personalities based on the psychometry of shopping, from preferences on brands, price points, etc. to activities and styles to emotions, and allows to incorporate decades of cognitive research into the learning system to reach out for a more generalized approach.
  • Using consumers’ natural-born value system: Meeting consumers’ wants at the deep and unconscious part (our emotions) is extremely effective in terms of neurobiological processing of the limbic brain. At the same time, the limbic brain is a consumer’s natural-born value system, and all decisions of the underlying “why” are treated here. Moreover, optimizing consumers’ emotional value is fully aligned with “good AI,” as it serves customers with what they really want even if they are not aware of it.
  • Structured data extraction—Building the deep value signature of products: The next step is to engineer a complete new breed of product data to create the above mentioned product personalities. Data is the key differentiator for AI and it is essential to develop new data preparation techniques to transform unstructured and dirty HTML data into cognified product metadata. ML is used to leverage some of the data preprocessing steps before NLP processing comes into play and text data is enriched with image data through graph-based machine learning. Then, techniques from cognitive and semantic science are applied to build the cognitive metadata layers.
  • DSRL—Using the best of both learning worlds: Deep symbolic reinforcement learning (DSRL) is a recently published method that combines the model-thinking approach with pattern recognition (i.e., a system that can learn representations and then perform higher-order reasoning about those representations). The technological component for this task is the relevance middleware (MindMachine), which matches the consumer personality with the product personalities. As we are using objects grounded in the psychometry of shopping (emotions, activities, etc.), the low-level symbol generation is obsolete. The next stage learns to track the objects in order to learn from their dynamics—applied by implementing a spatial sensor heat map that literally reflects consumer’s behavior by highlighting sensors. Finally, we enter the reinforcement learning stage, where the relative highlighting of sensors greatly reduces the spate space. On the assumption that single consumer behavior is random across the whole range of psychometric concepts, DSRL is a good choice, as pattern recognition only would always be sparse in order to form a statistical picture of all possible combinations.
  • Better perception of streamed behavioral sensor data: AI is very much about sensors. Deep shopping bots get noisy customer behavior data that must be transformed into meaningful signal data in real time. This can be done by implementing elaborate design patterns from reactive streaming analytics, a combination of complex event processing and streaming analytics. AI models work far better on consolidated states of belief, and you should ensure that kind of feature quality. This was solved by clearly organizing the various data streams of the single psychometric concepts in the streaming topology and implementing real-time data preparation techniques to get consolidated states on emotions, activities, etc.
  • The rise of AI interaction strategies: Strategies for interacting with the user could become a key field in next-gen AI systems and include playfulness and convenience, navigational concepts, and trust. For instance, it’s a good idea to make ecommerce AI more playful and entertaining. (This was discovered by unveiling a consumer’s personality through preference games and personality quizzes.) Second, bots should come with an appropriate navigation model. In order to make customers feel relaxed and safe on this journey, the bot should know where to go, proposing the structure and next best action. Finally, interactive bots should allow users to watch, edit, and reset the information collected about them. Transparency is achieved by putting the user into the control seat and providing a so-called “second self,” an avatar representing the most recent state of belief on consumer’s emotions, activities, styles, and preferences. This leaves the filter bubble behind and could be a major step toward an open and trustful conversation between users and interactive bots.
Photo of Rupert Steffner

Rupert Steffner


Rupert Steffner is the founder of WUNDER, a cognitive AI startup that is helping consumers find the products they love. Rupert has over 25 years of experience in designing and implementing highly sophisticated technical and business solutions, with a focus on customer-centric marketing. Previously, Rupert was chief platform architect of Otto Group’s new business intelligence platform BRAIN and head of BI at Groupon EMEA and APAC. He also served as business intelligence leader for several European and US companies in the ecommerce, retail, finance, and telco industries. He holds an MBA from WU Vienna and was head of the Marketing Department at the University of Applied Sciences in Salzburg.