Presented By O’Reilly and Intel Nervana
Put AI to work
September 17-18, 2017: Training
September 18-20, 2017: Tutorials & Conference
San Francisco, CA

In-Person Training
Natural language processing with deep learning

Delip Rao (AI Foundation)
Sunday, September 17 & Monday, September 18, 9:00am - 5:00pm
Location: Franciscan D
Average rating: *....
(1.00, 2 ratings)

Participants should plan to attend both days of this 2-day training course. Platinum and Training passes do not include access to tutorials on Monday.

Delip Rao explores natural language processing with deep learning, walking you through neural network architectures and NLP tasks and teaching you how to apply these architectures for those tasks.

What you'll learn, and how you can apply it

  • Understand basic concepts in natural language processing (NLP) and deep learning as they apply to NLP
  • Learn a hands-on approach to framing a real-world problem to the underlying NLP task and building a solution using deep learning

Prerequisites:

  • A working knowledge of Python and the command line
  • Familiarity with precalc math (multiply matrices, dot products of vectors, etc.) and derivatives of simple functions (If you are new to linear algebra, this video course is handy.)
  • A general understanding of machine learning (setting up experiments, evaluation, etc.) (useful but not required)

Hardware and/or installation requirements:

  • A laptop with the PyTorch environment set up (If you have trouble following the provided instructions or if you find any mistakes, please file an issue here.)

Natural language processing (NLP) involves the application of machine learning and other statistical techniques to derive insights from human language. With large volumes of data exchanged as text (in the form of documents, tweets, email, chat, and so on), NLP techniques are indispensable to modern intelligent applications. The applications range from enterprise to pedestrian.

Delip Rao explores natural language processing with deep learning, walking you through neural network architectures and NLP tasks and teaching you how to apply these architectures for those tasks.

Outline

Day 1

  • Environment setup and data download
  • Introduction to supervised learning
  • Introduction to computational graphs
  • Introduction to NLP and NLP tasks
  • Representations for words: Word embeddings
  • Hands-on exercise: Word analogy problems
  • Overview of deep learning frameworks
  • Static versus dynamic
  • PyTorch basics
  • Hands-on exercise: PyTorch
  • Feed-forward networks for NLP: Multilayer perceptrons
  • Hands-on exercise: Chinese document classification
  • Modeling subword units with convolutional networks
  • *Hands-on exercise: Classifying names to ethnicities


Day 2

Sequence modeling

  • Basics of modeling sequences
  • Representing sequences as tensors

Recurrent neural networks (RNNs) to model sequences

  • Basic concepts
  • Hands-on exercise: Language modeling using a character RNN
  • Gated variants (LSTM and GRU)
  • Structural variants (bidirectional, stacked, and tree)

Structured prediction methods

  • Greedy selection
  • Monte Carlo methods
  • Beam search
  • Viterbi decoding
  • Hands-on exercise: Generating sentences from a character RNN

Attention

  • Basic concepts
  • Applications: Context-aware modeling

From sequence models to sequence-to-sequence models

  • Basic ideas
  • Applications: Translation and summarization

Advanced topics

  • Memory networks
  • Convolutions for sequence modeling
  • Transfer learning
  • Multitask learning

DL modeling for common NLP tasks

  • Language modeling
  • POS tagging
  • Chunking
  • NER
  • Parsing
  • Machine translation
  • Summarization
  • Generation

Choose your own adventure

  • Hands-on exercise: Work with an NLP problem end-to-end from a selection of problems

DL for NLP: Best practices

Wrap-up and Q&A

  • When to use deep learning for NLP
  • When not to use deep learning for NLP

About your instructor

Photo of Delip Rao

Delip Rao is the founder of Joostware, a San Francisco-based company specializing in consulting and building IP in natural language processing and deep learning. Delip is a well-cited researcher in natural language processing and machine learning and has worked at Google Research, Twitter, and Amazon (Echo) on various NLP problems. He is interested in building cost-effective, state-of-the-art AI solutions that scale well. Delip has an upcoming book on NLP and deep learning from O’Reilly.

Twitter for deliprao

Conference registration

Get the Platinum pass or the Training pass to add this course to your package.

Comments on this page are now closed.

Comments

Ruth Toner | DATA SCIENTIST
09/14/2017 3:45pm PDT

I’ve set up the environment, cloned the repo, and installed all dependencies. Is there a link I’ve missed where we need to download the ~2GB of data ahead of time, or will we be downloading it during the class?

Alberto Odor | ADJUNCT PROFESSOR
09/14/2017 1:31am PDT

I have done all the installation of the required software for the course. Where should I unzip the tutorial file pytorch-nlp-tutorial-sf2017.zip?

Ahmed Chaudhary | SOFTWARE ENGINEER
09/09/2017 8:46am PDT

Hi Delip, I have the same question as Jack. Is there a plan to add more spots?

Jack Li | SOFTWARE ENGINEER
09/07/2017 12:23am PDT

Hi Delip, I saw the class is already full, is there a plan to accommodate more spaces?

Picture of Delip Rao
Delip Rao | VICE PRESIDENT, RESEARCH
09/05/2017 8:06am PDT

@Gautam, please make sure you have at least 2G free. Obviously, more the better.

Picture of Delip Rao
Delip Rao | VICE PRESIDENT, RESEARCH
09/05/2017 8:05am PDT

@Matthew, PyTorch does not have native windows support reliably yet. If you are using Windows, you have two options: 1) use a Linux virtual machine (VirtualBox or other options you get from googling “windows linux virtual machine”), 2) Use Docker. I believe the O’Reilly folks are writing a docker file in the works. When that happens, we will update the environment setup with additional details. So watch for that.

Matthew Reichenbach | BUSINESS INTELLIGENCE ARCHITECT
09/05/2017 7:12am PDT

Delip – thank you for the link regarding environment setup. I hate to be annoying, but would it be possible to provide a similar reference for Windows users? This link only provides assistance for OSX/Linux users. Thank you!

Gautam Karmakar | ARCHITECT
09/05/2017 7:10am PDT

Hi,
How much data will we be using? The reason I am asking because my laptop has limited space so want to make sure I keep enough space for data required in the training (I am not talking about model training).

Picture of Delip Rao
Delip Rao | VICE PRESIDENT, RESEARCH
09/03/2017 8:57am PDT

This page will be updated soon with environment setup and a revised outline, but for now, please follow the instructions in here for the environment setup.

Neeraj J | DATA ENGINEER
09/01/2017 6:14am PDT

Is this fully booked? Wondering if there is 1 more space . Please confirm.

Srinivas Reddy | DATA ENGINEER
08/26/2017 3:41am PDT

Hi, are there any prerequisites for this training like preinstalling software needed or any other preparatory steps?