Fairness through experimentation at LinkedIn
Who is this presentation for?Data scientists or analysts
Online controlled experiments have become a core decision-making tool in technology companies. However, most companies make decisions based on average effects, that is, focusing on whether a feature benefits the average user. This may be detrimental to the objectives of fairness, equality of opportunity, and social responsibility. For example, a product might benefit one subpopulation and hurt another.
Guillaume Saint-Jacques and Meg Garlinghouse show you how to use the Atkinson index in your A/B tests to measure whether the treatment increases inequality among users. They use real examples from LinkedIn, as well as a highly scalable implementation of the computation of the Atkinson index and its variance in Spark and Scala, which allows LinkedIn to compute the Atkinson index and its A/B variance for a sample of tens of millions in just 5–10 minutes.
A surprising finding is that many types of features are fairness enhancing, even if they weren’t explicitly designed to be. You’ll see examples of features that were fairness enhancing and features that weren’t, and you’ll learn how to compute LinkedIn’s fairness and inequality index on Spark, how to tell whether treatment and control are meaningfully different from a fairness perspective, and how to take action on the results inside your organization.
- A basic understanding of A/B testing
- General knowledge of statistics (useful but not required)
What you'll learn
- Understand that you can't purely focus on algorithmic fairness: people's behavioral reactions to products are important, experimentation is a very useful approach to fairness, computing inequality and entropy introduced by a feature can shed light on fairness impact, and these methods are easy to distribute and scale well on Spark
Guillaume Saint-Jacques is the tech lead of computational social science at LinkedIn. Previously, he was the technical lead of the LinkedIn experimentation science team. He holds a PhD in management research from the MIT Sloan School of Management, a master’s degree in economics from the Paris École Normale Supérieure and the Paris School of Economics, and a master’s degree in entrepreneurship from HEC Paris.
Meg Garlinghouse is the head of social impact at LinkedIn. She’s passionate about connecting people with opportunities to use their skills and experience to transform the world. She has more than 20 years of experience working at the intersection of nonprofits and corporations, developing strategic and mutually beneficial partnerships. She has particular expertise in leveraging media and technology to meet the marketing, communications, and brand goals of respective clients. Meg has a passion for developing innovative social campaigns that have a business benefit.
Leave a Comment or Question
Help us make this conference the best it can be for you. Have questions you'd like this speaker to address? Suggestions for issues that deserve extra attention? Feedback that you'd like to share with the speaker and other attendees?
Join the conversation here (requires login)
Premier Diamond Sponsors
Premier Exhibitor Plus
For conference registration information and customer service
For more information on community discounts and trade opportunities with O’Reilly conferences
For information on exhibiting or sponsoring a conference
For media/analyst press inquires