Improving infrastructure efficiency with unsupervised algorithms

Description

Water, sewage, energy, and public transportation systems in major cities are aging fast and are exceedingly difficult to update. Ceasing service to these critical utilities isn’t an option, as a mere hour’s delay can cause serious issues to residents and businesses in the area. The best way to improve these systems is to optimize their efficiency. However, the hardest part of optimizing infrastructure isn’t the algorithms or model creation but incorporating the necessary sensors and workflow adjustment into associated business processes.

Alexandre Hubert outlines a real-world example of energy sector optimization through automatic flagging of unnecessary field visits. in partnership with natural gas provider GRDF, Dataiku worked to optimize the manual process of qualifying addresses to visit and ultimately save GRDF time and money. Dataiku brought the system from an unsupervised problem with messy data into a successful automated process.

Alexandre discusses the challenges encountered along the way, including working with business experts to create a training set from disparate structured and unstructured data sources, and shares a model for how to implement unsupervised algorithms in a scalable way. This is the best first step in long-term infrastructure improvement—and a critical one as cities become more densely populated.