Contemporary search engines are often good at finding all results for a given query but less so at identifying which of these are the best results and thus should be returned first. Excellent recall is insufficient for useful search; search engines also need to identify the most relevant results in a sea of matches. Learning-to-rank algorithms aim to capture the relative utility of search results and thereby return useful suggestions quickly and efficiently. Learning to rank can be implemented with machine learning models of varying complexity, from standard linear regression to gradient-boosted decision trees.
Sophie Watson outlines some standard learning-to-rank methods and illustrates them by applying them to a real search engine. She compares off-the-shelf methods and discusses some of the pitfalls she ran into when training a learning-to-rank model and shares simple tips to improve results from the off-the-shelf methods, enabling you to build a robust and effective search engine.
You’ll walk away with an understanding of the problems involved in relevant search, an overview of key techniques, and the knowledge needed to implement learning-to-rank algorithms on your own dataset.
Sophie Watson is a senior data scientist at Red Hat, where she helps customers use machine learning to solve business problems in the hybrid cloud. She’s a frequent public speaker on topics including machine learning workflows on Kubernetes, recommendation engines, and machine learning for search. Sophie earned her PhD in Bayesian statistics.
For exhibition and sponsorship opportunities, email strataconf@oreilly.com
For information on trade opportunities with O'Reilly conferences, email partners@oreilly.com
View a complete list of Strata Data Conference contacts
©2019, O’Reilly UK Ltd • (800) 889-8969 or (707) 827-7019 • Monday-Friday 7:30am-5pm PT • All trademarks and registered trademarks appearing on oreilly.com are the property of their respective owners. • confreg@oreilly.com