Handtrack.js: Building Gesture Based Interactions in the Browser Using Tensorflow.js
What you'll learn
In the browser, Machine Learning can enable truly novel forms of interactions, while reaping the benefits associated with on-device computation such as reduced latency for interactive applications, reduced model distribution costs, and enhanced privacy as data is no longer sent to remote servers for analysis.
In this talk, I provide an overview of the Tensorflow.js library, benchmark performance results for image tasks in the browser and cover an end-to-end example on my experience building Handtrack.js – a library for prototyping real time hand tracking interactions. Handtrack.js is powered by an object detection neural network (MobilenetV2, SSD) and allows users predict the location (bounding box) of human hands in an image, video or canvas html tag. The talk will cover steps and best practices for deploying a neural network model in the browser – from data collection, model training, model conversion to Tensorflow.js webmodel format, model hosting and inference. A live demo prototyped using Handtrack.js will be shown.
Cloudera Fast Forward Labs
Victor Dibia is a Research Engineer with Cloudera’s Fast Forward Labs where his work focuses on prototyping state of the art machine learning algorithms and advising clients. Prior to this, he was a Research Staff Member at the IBM TJ Watson Research Center, New York. His research interests are at the intersection of human computer interaction, computational social science, and applied AI. He holds an M.S. from Carnegie Mellon University and a Ph.D. from City University of Hong Kong.
Leave a Comment or Question
Help us make this conference the best it can be for you. Have questions you'd like this speaker to address? Suggestions for issues that deserve extra attention? Feedback that you'd like to share with the speaker and other attendees?
Join the conversation here (requires login)
For conference registration information and customer service
For more information on community discounts and trade opportunities with O’Reilly conferences
For information on exhibiting or sponsoring a conference
View a complete list of Strata Data Conference contacts