Unlocking insights in AI by building a feature store

Description

GOJEK, Indonesia’s first billion-dollar startup, has seen an explosive growth in both users and data over the past three years. Today, it uses big data-powered machine learning to inform decision making in its ride-hailing, lifestyle, logistics, food delivery, and payment products, from selecting the right driver to dispatch to dynamically setting prices to serving food recommendations to forecasting real-world events. Hundreds of millions of orders per month, across 18 products, are all driven by machine learning.

Features are at the heart of what makes these machine learning systems effective. However, many challenges still exist in the feature lifecycle. Developing features from big data is often an engineering heavy task, with challenges in both the scaling of data processes and the serving of features in production systems. Teams also face challenges in enabling discovery, reducing duplication, improving understanding, and providing standardization of features throughout organizations.

Willem Pienaar and Zhiling Chen explain the need for features at organizations like GOJEK and discuss the challenges faced in creating, managing, and serving them in production. They describe how in partnership with Google, they designed and built a feature store called Feast to address these challenges and explore their motivations, the lessons they learned along the way, and the impact the feature store had on GOJEK. Finally, they demonstrate how democratizing the process of creating, sharing, and managing features dramatically reduces time to market and leads to key insights.