Presented By O’Reilly and Intel AI
Put AI to Work
April 29-30, 2018: Training
April 30-May 2, 2018: Tutorials & Conference
New York, NY

In-Person Training
Natural language processing with deep learning

Delip Rao (R7 Speech Science), Brian McMahan (Joostware)
9:00am–5:00pm
Sunday, April 29 through Monday, April 30
Location: Clinton
SOLD OUT
Average rating: ***..
(3.83, 6 ratings)

Participants should plan to attend both days of this 2-day training course. Platinum and Training passes do not include access to tutorials on Monday.

Delip Rao explores natural language processing with deep learning, walking you through neural network architectures and NLP tasks and teaching you how to apply these architectures for those tasks.

What you'll learn, and how you can apply it

  • Understand basic concepts in natural language processing (NLP) and deep learning as they apply to NLP
  • Learn a hands-on approach to framing a real-world problem to the underlying NLP task and building a solution using deep learning

Prerequisites:

  • A working knowledge of Python and the command line
  • Familiarity with precalc math (multiply matrices, dot products of vectors, etc.) and derivatives of simple functions (If you are new to linear algebra, this video course is handy.)
  • A general understanding of machine learning (setting up experiments, evaluation, etc.) (useful but not required)

Hardware and/or installation requirements:

  • A laptop with the PyTorch environment set up (If you have trouble following the provided instructions or if you find any mistakes, please file an issue here.)

Natural language processing (NLP) involves the application of machine learning and other statistical techniques to derive insights from human language. With large volumes of data exchanged as text (in the form of documents, tweets, email, chat, and so on), NLP techniques are indispensable to modern intelligent applications. The applications range from enterprise to pedestrian.

Delip Rao explores natural language processing with deep learning, walking you through neural network architectures and NLP tasks and teaching you how to apply these architectures for those tasks.

Outline

Day 1

Environment setup and data download

Fundamentals

  • Introduction to supervised learning
  • Introduction to computational graphs
  • Introduction to NLP and NLP tasks

Representations for words: Word embeddings

  • Overview of embeddings
  • Hands-on exercise: Word analogy problems

Deep learning frameworks

  • Static versus dynamic
  • PyTorch basics
  • Hands-on exercise: PyTorch

Feed-forward networks for NLP

  • Multilayer perceptrons
  • Hands-on exercise: Chinese document classification

Modeling subword units with convolutional networks

  • Hands-on exercise: Classifying names to ethnicities


Day 2

Sequence modeling

  • Basics of modeling sequences
  • Representing sequences as tensors

Recurrent neural networks (RNNs) to model sequences

  • Basic concepts
  • Hands-on exercise: Language modeling using a character RNN
  • Gated variants (LSTM and GRU)
  • Structural variants (bidirectional, stacked, and tree)

Structured prediction methods

  • Greedy selection
  • Monte Carlo methods
  • Beam search
  • Viterbi decoding
  • Hands-on exercise: Generating sentences from a character RNN

Attention

  • Basic concepts
  • Applications: Context-aware modeling

From sequence models to sequence-to-sequence models

  • Basic ideas
  • Applications: Translation and summarization

Advanced topics

  • Memory networks
  • Convolutions for sequence modeling
  • Transfer learning
  • Multitask learning

DL modeling for common NLP tasks

  • Language modeling
  • POS tagging
  • Chunking
  • NER
  • Parsing
  • Machine translation
  • Summarization
  • Generation

Choose your own adventure

  • Hands-on exercise: Work with an NLP problem end-to-end from a selection of problems

DL for NLP: Best practices

Wrap-up and Q&A

  • When to use deep learning for NLP
  • When not to use deep learning for NLP

About your instructors

Photo of Delip Rao

Delip Rao is the founder of R7 Speech Science, a San Francisco-based company focused on building innovative products on spoken conversations. Previously, Delip was the founder of Joostware, which specialized in consulting and building IP in natural language processing and deep learning. Delip is a well-cited researcher in natural language processing and machine learning and has worked at Google Research, Twitter, and Amazon (Echo) on various NLP problems. He is interested in building cost-effective, state-of-the-art AI solutions that scale well. Delip has an upcoming book on NLP and deep learning from O’Reilly.

Twitter for deliprao

Brian McMahan is a research engineer at Joostware, a San Francisco-based company specialized in consulting and building intellectual property in natural language processing and deep learning. He is also a cofounder at R7 Speech Sciences, a company focused on understanding spoken conversations. Brian is wrapping up his PhD in computer science from Rutgers University, where his research focuses on Bayesian and deep learning models for grounding perceptual language in the visual domain. Brian has also conducted research in reinforcement learning and various aspects of dialogue systems.

Conference registration

Get the Platinum pass or the Training pass to add this course to your package.

Comments on this page are now closed.

Comments

joe mcduffey | CEO
05/03/2018 11:12am EDT

Can I get the slides for the 2 day course?

Rovina Coutinho | BUSINESS ANALYST
05/03/2018 5:46am EDT

It would have been helpful if the session listed accurate pre-requisites. Though I had linear algebra and python background, I was not able to follow this session as it was more advanced than I expected.