On the accountability of black boxes: How to control what you can’t exactly measure

There’s little doubt that algorithmic systems are making decisions that have a great impact on our daily lives. Their authority is increasingly expressed algorithmically, and decisions that used to be based on human intuition and reflection are now automated, so transparency οn how these systems work matters not as an end in itself but merely as a means of accountability.

Trying to render an algorithmic system accountable means that a series of challenges need to be addressed. For instance, how do you keep a balance between high precision and transparency? Deep learning models are well-known for the former, not the latter. Also, organizations tend to keep their algorithms secret, claiming they want to preserve valuable intellectual property or avoid the risk of them getting gamed (e.g., in the case of credit scoring algorithms). Finally, there’s no widely accepted industry standard that defines how an algorithmic system should be evaluated in terms of transparency and accountability.

Yiannis Kanellopoulos shares an evaluation framework that reflects the state of practice as applied in several organizations. The framework is based on the thesis that data is socially constructed, so it covers both the algorithms themselves and the organizations that utilize them and need to cater for their accountability. It’s domain agnostic, so it can be operationalized at any type of organization, business domain, and type of algorithm, and it’s not intrusive, as it consists of a set of questions that require experts’ input. Yiannis then highlights lessons learned from the framework’s actual operationalization at a multibillion dollar high-tech corporation.