We run, we improve, we scale: The XGBoost story at Uber

Description

With the tremendous growth of Uber’s business scale, the agility and scalability of the machine learning system is the core prerequisite in making data-driven decisions to improve user experiences. With a good fitting to Uber’s requirements, XGBoost plays multiple roles across the business scope. XGBoost not only produces accurate models but also scales to handle billions of records and thousands of features. XGBoost models improve the driver’s safety during driving, recommends foods and restaurants, estimates the arrival time of rides, etc.

Nan Zhu and Felix Cheung share their insights about the internals of how XGBoost scales training to hundreds, even thousands, of workers with the accuracy guarantee. This is the first time that a community core member brings detailed internals of distributed training to a public audience. They also detail Uber’s journey with the latest version of XGBoost, including the problems the company had with the earlier version of XGBoost, how it identifies, fixes, and eventually unblocks itself by improving XGBoost and contributing back to the community. You’ll leave with a summary of lessons Uber learned and insight into its future plans.