Put AI to work
June 26-27, 2017: Training
June 27-29, 2017: Tutorials & Conference
New York, NY

Deep Shopping bots – Building machines that think and sell like humans

Rupert Steffner (WUNDERAI GmbH)
4:50pm5:30pm Thursday, June 29, 2017
Interacting with AI
Location: Murray Hill E/W Level: Intermediate
Secondary topics:  Machine Learning, Retail and e-commerce, User interface and experience

Prerequisite Knowledge

Basics in AI, Deep Reinforcement Learning, Model-thinking

What you'll learn

If you are planning to build and launch a smart bot for interacting with customers, the right data, algorithms, technology and value system are what you should be concerned with and prepared for. So how exactly can you design this kind of AI bot in terms of logical and technological components? And what are the cornerstones of real-world AI applications interacting with customers.


In the coming years we will see various kinds of machines thinking and learning like humans – from driving and working to interacting like humans. This presentation will focus specifically on the cornerstones of building smart interactive bots for selling and why they will do a far better job compared to online selling happening today. In an attention economy, shopping bots that help overcome choice or information overload have to become more sophisticated. Deep Shopping bots like the one presented here are based on true personal relevance and ease & joy of shopping – they try to read customer’s mind, know all the products in the market, are playful and conversational, and act and learn in real-time.

Layering models for a more generalized AI approach
Specific models often result from trying to put everything into one model. Instead, we combine two models to achieve a more generic Cognitive AI approach: The first one is a high-level model that goes for the true personal relevance shopping context by matching Consumer personalities directly against Product personalities. The second one goes for the deep level (Deep Shopping) by defining the personalities based on the psychometry of Shopping – from preferences on brands, price points etc. to activities and styles to emotions – and allows to incorporate decades of cognitive research into the learning system to reach out for a more generalized approach.

Using consumer’s natural born value system
Meeting consumer’s wants at the deep and unconscious part – our emotions – is extremely effective in terms of neuro-biological processing of the limbic brain. At the same time, the limbic brain with its emotions is consumer’s natural born value system and all decisions of the underlying ‘why’ are treated here. Moreover, optimizing consumer’s emotional value is fully aligned with ‘Good AI’ as it serves customers with what they really want even if they are not aware of it.

Structured data extraction – Building the deep value signature of products
Next step is to engineer a complete new breed of product data to create the above mentioned Product personalities. Data is the key differentiator for AI and it is essential to develop new data preparation techniques to transform unstructured and dirty HTML data into cognified product metadata. ML is used to leverage some of the data pre-processing steps before NLP processing comes into play and text data is enriched with image data through graph-based machine learning. Then, techniques from cognitive and semantic science are applied to build the cognitive metadata layers.

DSRL – Using the best of both learning worlds
Deep symbolic reinforcement learning (DSRL) is a recently published method which we use to combine the model-thinking approach with pattern recognition, i.e. a system that can learn representations and then perform higher-order reasoning about those representations. The technological component for this task is the relevance middleware (MindMachine) which is matching the Consumer personality with the Product personalities. As we are using objects grounded in the psychometry of shopping (emotions, activities etc.), the low-level symbol generation is obsolete. The next stage learns to track the objects in order to learn from their dynamics. We applied this by implementing a spatial sensor heat map that literally reflects consumer’s behavior by highlighting sensors. Finally, we enter the reinforcement learning stage where the relative highlighting of sensors greatly reduces the spate space. On the assumption that single consumer behavior is random across the whole range of psychometric concepts, DSRL is a good choice as pattern recognition only would always be sparse in order to form a statistical picture of all possible combinations.

Better perception of streamed behavioral sensor data
AI is very much about sensors. Deep Shopping bots get noisy customer behavior data that must be transformed into meaningful signal data in real-time. This can be done by implementing elaborated design patterns from reactive streaming analytics, a combination of complex event processing and streaming analytics. AI models work far better on consolidated states of belief and you should ensure that kind of feature quality. We solved this by clearly organizing the various data streams of the single psychometric concepts in the streaming topology and implementing real-time data preparation techniques to get consolidated states on emotions, activities etc.

The rise of AI interaction strategies
Strategies for interacting with the user could become a key field in next-gen AI systems. They will consider playfulness and convenience, navigational concepts and trust. For E-commerce AI, it’s a good idea to make it more playful and entertaining. We solved it by unveiling consumer’s personality through preference games and personality quizzes. Second, bots should come with an appropriate navigation model for which we developed a hub & flow navigation very close to how games are building their navigation. One of the new things for consumers will be about discovery and creating serendipitous moments. In order to let customers feel relaxed but safe on this journey, the bot should know where to go proposing the structure and next-best action. Finally, interactive bots should allow users to watch, edit and reset the information they collected about them. We have applied transparency and putting the user into the control seat by providing the so-called ‘Second self’ – an avatar representing the most recent state of belief on consumer’s emotions, activities, styles and preferences. This is leaving the filter bubble behind and could be a major step towards an open and trustful conversation between users and interactive bots.

Photo of Rupert Steffner

Rupert Steffner


Rupert Steffner is the founder of WUNDER.AI – an E-commerce AI start-up that is giving customers a joy ride to the products they love. Before that Rupert has been Chief Platform Architect of Otto Group’s new Business Intelligence Platform BRAIN and Head of BI at Groupon EMEA&APAC. He worked for several European and US companies as Business Intelligence Leader (e-commerce, retail, finance, telco). Rupert has over 25 years of experience in designing and implementing highly sophisticated technical and business solutions with a focus on customer centric marketing. He holds a Master of Business Administration of the WU Vienna and was Head of Marketing Department at the University of Applied Sciences, Salzburg.

Leave a Comment or Question

Help us make this conference the best it can be for you. Have questions you'd like this speaker to address? Suggestions for issues that deserve extra attention? Feedback that you'd like to share with the speaker and other attendees?

Join the conversation here (requires login)