The science of patchy data

Description

In the current golden days of big data, it seems that data scientists have unlimited access to all the data they need to create always better performing models and data products, and artificial intelligence is starting to outperform humans in multiple areas. However, the race to collect data is also raising an awareness of how this data relates to concerns about privacy and intellectual property. While the amount of data we collect is constantly growing, the challenges related to its fullness and accessibility are now requiring the big data community to become ever more creative and find new ways to extract value from it while respecting users’ right to privacy. This phenomenon is spurring what can be appropriately called “data science of the gaps.” Developing predictive models is now like putting a jigsaw puzzle together—trying to establish the big picture while potentially missing some essential pieces.

As a leader in the software industry, Atlassian takes this matter very seriously and is developing various ways to design intelligent features for its products while ensuring that its customers’ privacy is respected. Jennifer Prendki explains how Atlassian develops machine learning models even if the data is protected by privacy and compliance laws and cannot be used without anonymizing, covering techniques ranging from contextual bandits to document vector representation.