Mar 15–18, 2020

Natural language processing with deep learning

Delip Rao (AI Foundation)
9:00am—5:00pm
Sunday, March 15—Monday, March 16
Location: 114

Participants should plan to attend both days of training course. Note: to attend training courses, you must be registered for a Platinum or Training pass; does not include access to tutorials on Monday.

Delip Rao explores natural language processing (NLP) using a set of machine learning techniques known as deep learning. He walks you through neural network architectures and NLP tasks and teaches you how to apply these architectures for those tasks.

What you'll learn, and how you can apply it

  • Understand basic concepts in NLP and deep learning as it applies to NLP
  • Gain a hands-on approach to framing a real-world problem to the underlying NLP task and building a solution using deep learning

Prerequisites:

  • A working knowledge of Python and command-line familiarity
  • Familiarity with precalc math (multiply matrices, dot products of vectors) and derivatives of simple functions (useful but not required)
  • General knowledge of machine learning (setting up experiments, evaluation, etc.) (useful but not required)

Hardware and/or installation requirements:

  • A WiFi-enabled laptop

Outline

NLP involves the application of machine learning and other statistical techniques to derive insights from human language. With large volumes of data exchanged as text (in the form of documents, tweets, email, chat, and so on), NLP techniques are indispensable to modern intelligent applications. The applications range from enterprise to pedestrian.

Day 1

  • Environment set up and data download
  • Introduction to supervised learning
  • Introduction to computational graphs
  • Introduction to NLP and NLP tasks
  • Representations for words: Word embeddings
    • Hands-on: Word analogy problems
  • Overview of deep learning frameworks
  • Static versus dynamic
  • PyTorch basics
    • Hands-on: PyTorch exercises
  • Feed-forward networks for NLP: Multilayer perceptrons
    • Hands-on: Chinese document classification
  • Convolutional networks: Modeling subword units
    • Hands-on: Classifying names to ethnicities

Day 2

  • Sequence modeling: Basics of modeling sequences, representing sequences as tensors, the importance of the language modeling task
  • Recurrent neural networks (RNNs) to model sequences: Basic ideas
    • Hands-on: Classification with an RNN
    • Gated variants (long short-term memory (LSTM) and gated recurrent unit (GRU))
    • Structural variants (bidirectional, stacked, tree)
    • Hands-on: Generating sequences with an RNN
  • From sequence models to sequence-to-sequence models: Core ideas, encoder-decoder architectures, applications—translation and summarization
  • Attention: Core ideas and its role in Seq2Seq models
  • Advanced topics
    • Self-attention and the Transformer
    • Contextualized embedding models: BERT, ELMo
      • Hands-on: BERT
  • Overview of DL modeling for common NLP tasks
  • Choose your own adventure
    • Hands-on: Work with an NLP problem end-to-end from a selection of tasks
  • DL for NLP: Best practices
  • When to use deep learning for NLP, when not to use deep learning for NLP, and summary

About your instructor

Delip Rao is the vice president of research at the AI Foundation, where he leads speech, language, and vision research efforts for generating and detecting artificial content. Previously, he founded the AI research consulting company Joostware and the Fake News Challenge, an initiative to bring AI researchers across the world together to work on fact checking-related problems, and he was at Google and Twitter. Delip is the author of a recent book on deep learning and natural language processing. His attitude toward production NLP research is shaped by the time he spent at Joostware working for enterprise clients, as the first machine learning researcher on the Twitter antispam team, and as an early researcher at Amazon Alexa.

Twitter for deliprao

Conference registration

Get the Platinum pass or the Training pass to add this course to your package. Early Price ends February 7.

Leave a Comment or Question

Help us make this conference the best it can be for you. Have questions you'd like this speaker to address? Suggestions for issues that deserve extra attention? Feedback that you'd like to share with the speaker and other attendees?

Join the conversation here (requires login)

Contact us

confreg@oreilly.com

For conference registration information and customer service

partners@oreilly.com

For more information on community discounts and trade opportunities with O’Reilly conferences

Become a sponsor

For information on exhibiting or sponsoring a conference

pr@oreilly.com

For media/analyst press inquires