Put AI to Work
April 15-18, 2019
New York, NY

Automotive AI and drowsiness state monitoring

Mohammad Mavadati (Affectiva)
4:55pm5:35pm Thursday, April 18, 2019
Case Studies, Machine Learning
Location: Sutton South
Secondary topics:  Models and Methods, Reliability and Safety, Temporal data and time-series

Who is this presentation for?

  • AI engineers and machine learning scientists



Prerequisite knowledge

  • A basic understanding of machine learning and deep neural networks

What you'll learn

  • Explore real-world drowsy in-car data
  • Learn how to utilize computer vision and AI technology to model the intensity of drowsiness


The automotive industry has been striving to utilize AI to track the exterior activities of the car and therefore be more aware of the driving contexts. Affectiva’s mission is to extend its Affectiva Automotive AI technology for use cases related to improving driver safety. Affectiva Automotive AI has been and will be one of the critical pillars of a user-friendly, safe in-cabin experience for passengers.

According to the CDC, drowsy drivers cause up to 6,000 fatal crashes annually. AI is a powerful, viable approach to monitoring states of drivers and reducing drowsy driving risks. Mohammad Mavadati explores Affectiva’s in-car drowsiness dataset and the annotation infrastructures for labeling driver video data before demonstrating how the merger of facial expression modeling and machine learning techniques can be used to predict, monitor, and track drivers drowsiness states. Using automatic drowsiness state monitoring system can effectively assess and approximate alertness levels of a person using facial expression patterns over time.

Mohammad outlines Affectiva’s approach to developing AI systems that can observe such states through using spatiotemporal models using its data labeling methodology. He then compares the performance of DNN models with existing models approaches and shares augmentation and data sampling strategies that improve the quality of models and utilize the drowsy dataset effectively. He also discusses some of the challenges (e.g., imbalanced representation of drowsy states, uneven video sampling rates, and partial facial tracking) and related solutions for in-car data as well as a variety of architectures (e.g., CNN and CNN+RNN) for drowsiness intensity prediction. Mohammad concludes with a demo that illustrates how this technology performs in real time.

Photo of Mohammad Mavadati

Mohammad Mavadati


Mohammad Mavadati is lead computer vision scientist at Affectiva, where he leads the computer vision team to deliver state-of-the-art emotion-aware AI technology. Mohammad’s mission is to bring emotional intelligence to human-machine interaction and utilize his extensive research and affective-computing expertise to deliver smarter and more emotionally aware services to the automotive industry. He has coauthored more than 20 technical publications related to affective computing. Mohammad holds a PhD in computer science from University of Denver.

Leave a Comment or Question

Help us make this conference the best it can be for you. Have questions you'd like this speaker to address? Suggestions for issues that deserve extra attention? Feedback that you'd like to share with the speaker and other attendees?

Join the conversation here (requires login)