Presented By
O’Reilly + Cloudera
Make Data Work
March 25-28, 2019
San Francisco, CA
Please log in

Framework to quantitatively assess ML safety: Technical implementation and best practices

Ram Shankar Siva Kumar (Microsoft (Azure Security))
11:00am–11:40am Thursday, March 28, 2019
Data Science, Machine Learning & AI
 Location: 2010
Secondary topics:  AI and machine learning in the enterprise, Health and Medicine, Security and Privacy
Average rating: ****.
(4.33, 3 ratings)

Who is this presentation for?

  • Data scientists, security experts, and program managers focused on data-related products

Level

Intermediate

Prerequisite knowledge

  • Experience as a data scientist (useful but not required)

What you'll learn

  • Explore a framework to quantitatively assess the safety rating of machine learning systems
  • Get best practices for calculating safety rating as well as an action plan broken down into monthly and quarterly goals
  • Dive into sample code (in IPython Notebook) that you can modify for your ML system

Description

Machine learning is already at the core of many critical systems including healthcare, cybersecurity, finance, and transportation. The papers on adversarial machine learning are piling up in arXiv, but what would a system that assess the safety of ML system look like in practice? What does it mean for data scientists to guarantee that their system is adequately protected from adversarial manipulation?

Ram Shankar Kumar shares a framework and corresponding best practices to quantitatively assess the safety of your ML systems. The opportunities when such a framework is put to effect are plentiful; for a start, you regain your customers’ trust that ML systems aren’t brittle; that they just come in varying, quantifiable degrees of safety.

This talk represents work from Azure Security Data Science and Microsoft Research and work done at the Berkman Klein Center at Harvard University.

Topics include:

  • Why we need such a framework
  • How to measure safety of an ML system
  • How do you know if a system is “safe enough”?
  • Who should be calculating the safety rating?
  • When should the rating be calculated?
  • How often should the rating be calculated?
  • Known limitations
  • Demonstration of the framework in action (to calculate the safety rating of a ResNet model trained on ImageNet, susceptible to blackbox attacks)

Ram Shankar Siva Kumar

Microsoft (Azure Security)

Ram Shankar is a data cowboy on the Azure security data science team at Microsoft, where his team focuses on modeling massive amounts of security logs to surface malicious activity. His work has appeared in industry conferences like DEF CON, BSides, BlueHat, DerbyCon, MIRCon, Infiltrate, and Strata as well as academic conferences like NIPS and ACM-CCS. Ram holds a degree focused on machine learning and security from Carnegie Mellon University. He’s currently an affiliate at the Berkman Klein Center at Harvard, exploring the intersection of machine learning and security.