Presented By O’Reilly and Intel AI
Put AI to work
8-9 Oct 2018: Training
9-11 Oct 2018: Tutorials & Conference
London, UK

Multitask learning in PyTorch applied to news classification

Ryan Micallef (Cloudera Fast Forward Labs)
11:05–11:45 Thursday, 11 October 2018
Models and Methods
 Location: Hilton Meeting Room 3-6
Secondary topics:  Media, Marketing, Advertising, Text, Language, and Speech

Who is this presentation for?

  • Data scientists, machine learning engineers, and data product managers

Prerequisite knowledge

  • A general understanding of how supervised algorithms work
  • Familiarity with neural networks

What you'll learn

  • Understand multitask learning, an approach to problem solving that allows supervised algorithms to be trained to master more than one objective at once and in parallel with applications in natural language processing, computer vision, and healthcare
  • Explore regularization techniques, including adversarial training, that allow networks to learn shared, task-agnostic representations, with respect to the tasks that are part of the training, that are better suited for adapting a model to a new task not part of the original training
  • Discover how private-shared component analysis enables insights into the relatedness of tasks part of the multitask learning setup
  • See why PyTorch offers an excellent framework for implementing multitask networks (including examples of layers, models, and loss functions)

Description

Multitask learning offers an approach to problem solving that allows supervised algorithms to master more than one objective (or task) at once and in parallel. Ryan Micallef offers an overview of opportunities for multitask learning in natural language processing, computer vision, and healthcare and shares a neural network, implemented in PyTorch, that is trained to classify news articles using multitask learning. You’ll learn how to implement custom layers, models, and loss functions in PyTorch and explore private-share component analysis.

Multitask training, combined with regularization techniques such as adversarial training, allows neural networks to learn task-specific and task-agnostic representations (with respect to the tasks part of the multitask training setup). The analysis of these representations (i.e., private-shared component analysis) allows insights into how task relate to each other. The prototype demonstrates the use of publication-specific language and language shared across publication in articles of different types, from politics to sports and economics, based on multitask learning and private-shared component analysis.

Photo of Ryan Micallef

Ryan Micallef

Cloudera Fast Forward Labs

Ryan Micallef is a research engineer at Cloudera Fast Forward Labs focused on studying emerging machine learning technologies and helping clients apply them. Ryan is also an attorney barred in New York and spent nearly a decade as an intellectual property litigator focused on technical cases. Ryan holds a bachelor’s degree in computer science from Georgia Tech and a JD from Brooklyn Law School. He spends his free time soldering circuits and wrenching motorcycles. He also teaches microcontroller programming at his local hackerspace, NYC Resistor.