Put AI to Work
April 15-18, 2019
New York, NY
Please log in

In-Person Training
Natural language processing with deep learning (SOLD OUT)

Delip Rao (AI Foundation), Brian McMahan (Wells Fargo)
Monday, April 15 & Tuesday, April 16,
9:00am - 5:00pm
Implementing AI, Models and Methods
 Location: Clinton
Secondary topics:  Deep Learning and Machine Learning tools, Models and Methods, Text, Language, and Speech
SOLD OUT
Average rating: ****.
(4.33, 3 ratings)

Participants should plan to attend both days of this 2-day training course. To attend training courses, you must register for a Platinum or Training pass; does not include access to tutorials on Tuesday.

Delip Rao and Brian McMahan explore natural language processing with deep learning, walking you through neural network architectures and NLP tasks and teaching you how to apply these architectures for those tasks.

What you'll learn, and how you can apply it

  • Understand basic concepts in natural language processing (NLP) and deep learning
  • Learn a hands-on approach to framing a real-world problem to the underlying NLP task and building a solution using deep learning

Prerequisites:

  • A working knowledge of Python and the command line
  • Familiarity with precalc math (multiply matrices, dot products of vectors, etc.) and derivatives of simple functions (If you are new to linear algebra, this video course is handy.)
  • A general understanding of machine learning (setting up experiments, evaluation, etc.) (useful but not required)

Hardware and/or installation requirements:

  • You will need a moderately capable laptop or tablet with keyboard with you. Please make sure you have one!
  • There is no installation or OS requirements as all coding sessions will happen on JupyterHub, but you will need a modern browser to access that.
  • Each participant will have a unique JupyterHub login and will have a finite quota of compute hours.
  • We recommend the participants to save all their work locally by downloading any modified notebooks, and turning off the notebook kernels by end of day
    (don't worry, we will go over this in class)

Natural language processing (NLP) involves the application of machine learning and other statistical techniques to derive insights from human language. With large volumes of data exchanged as text (in the form of documents, tweets, email, chat, and so on), NLP techniques are indispensable to modern intelligent applications. The applications range from enterprise to pedestrian.

Delip Rao and Brian McMahan explore natural language processing with deep learning, walking you through neural network architectures and NLP tasks and teaching you how to apply these architectures for those tasks.

Outline

Day 1

Environment setup and data download

Fundamentals

  • Introduction to supervised learning
  • Introduction to computational graphs
  • Introduction to NLP and NLP tasks

Representations for words: Word embeddings

  • Overview of embeddings
  • Hands-on exercise: Word analogy problems

Deep learning frameworks

  • Static versus dynamic
  • PyTorch basics
  • Hands-on exercise: PyTorch

Feed-forward networks for NLP

  • Multilayer perceptrons
  • Hands-on exercise: Chinese document classification

Modeling subword units with convolutional networks

  • Hands-on exercise: Classifying names to ethnicities


Day 2

Sequence modeling

  • Basics of modeling sequences
  • Representing sequences as tensors

Recurrent neural networks (RNNs) to model sequences

  • Basic concepts
  • Hands-on exercise: Language modeling using a character RNN
  • Gated variants (LSTM and GRU)
  • Structural variants (bidirectional, stacked, and tree)

Structured prediction methods

  • Greedy selection
  • Monte Carlo methods
  • Beam search
  • Viterbi decoding
  • Hands-on exercise: Generating sentences from a character RNN

Attention

  • Basic concepts
  • Applications: Context-aware modeling

From sequence models to sequence-to-sequence models

  • Basic ideas
  • Applications: Translation and summarization

Advanced topics

  • Memory networks
  • Convolutions for sequence modeling
  • Transfer learning
  • Multitask learning

DL modeling for common NLP tasks

  • Language modeling
  • POS tagging
  • Chunking
  • NER
  • Parsing
  • Machine translation
  • Summarization
  • Generation

Choose your own adventure

  • Hands-on exercise: Work with an NLP problem end-to-end from a selection of problems

DL for NLP: Best practices

Wrap-up and Q&A

  • When to use deep learning for NLP
  • When not to use deep learning for NLP

About your instructors

Photo of Delip Rao

Delip Rao is the vice president of research at the AI Foundation, where he leads speech, language, and vision research efforts for generating and detecting artificial content. Previously, he founded the AI research consulting company Joostware and the Fake News Challenge, an initiative to bring AI researchers across the world together to work on fact checking-related problems, and he was at Google and Twitter. Delip is the author of a recent book on deep learning and natural language processing. His attitude toward production NLP research is shaped by the time he spent at Joostware working for enterprise clients, as the first machine learning researcher on the Twitter antispam team, and as an early researcher at Amazon Alexa.

Twitter for deliprao

Brian McMahan is a data scientist at Wells Fargo, working on projects that apply natural language processing (NLP) to solve real world needs. Recently, he published a book with Delip Rao on PyTorch and NLP. Previously, he was a research engineer at Joostware, a San Francisco-based company specializing in consulting and building intellectual property in NLP and Deep Learning. Brian is wrapping up his PhD in computer science from Rutgers University, where his research focuses on Bayesian and deep learning models for grounding perceptual language in the visual domain. Brian has also conducted research in reinforcement learning and various aspects of dialogue systems.

Conference registration

Get the Platinum pass or the Training pass to add this course to your package.

Comments on this page are now closed.

Comments

Picture of Adel Youssef
Adel Youssef | RESEARCH MANAGER - NLP
04/21/2019 1:44pm EDT

Hello Delip/Brian,
When should we expect to receive the course materials (slides, Jupyter notebooks)?
Thank you
P.S. I signed up already on the Google form using the link given in the course.

. . |
04/15/2019 5:50am EDT

Hi, I am for another training and cannot attend NLP :( Will be you support materials available somehow for conference participants?
Thanks

Brian McMahan | RESEARCH SCIENTIST
04/13/2019 3:28pm EDT

I think windows compatibility might be working now. If you are able to get it working, that is great! Just as long as you are aware that there is a potential risk involved (if, for example, some features of the library don’t work with Windows), then it should be fine. O’Reilly will be providing a Jupyter instance for each attendee as well, so that is always there as backup.

Don Brasek | TECHNICAL LEAD
04/04/2019 11:27am EDT

The environment setup instructions indicate that we cannot use Windows directly. However, after some tweaking (0-dim tensor indexing; specifying utf8 encoding when reading in data files) I was able to get all the joosthub/pytorch-nlp-tutorial-nyc2017 notebooks working on my Windows 10 laptop. Or at least I think it’s all working—I’m a beginner. Anything in particular I should double-check, or is there other material that definitely won’t work? I typically prefer my macbook (and its GPU is CUDA compatible) but this windows laptop is light! :-)

Marina Marin | ASSOC. DIRECTOR, INFORMATION MANAGEMENT
03/25/2019 5:45am EDT

Looks like this course is sold out. Is there any way the number of spots could be increased?
Thank you!

kaushik Narayanan | SOFTWARE ENGINEERING MANAGER
02/22/2019 11:52am EST

what background do I need? I am a software engineering manager. Do I only need basics of Python? I don’t have much background in ML