Natural language processing with deep learning (Day 2)

Natural language processing (NLP) involves the application of machine learning and other statistical techniques to derive insights from human language. With large volumes of data exchanged as text (in the form of documents, tweets, email, chat, and so on), NLP techniques are indispensable to modern intelligent applications. The applications range from enterprise to pedestrian.

Delip Rao explores natural language processing with deep learning, walking you through neural network architectures and NLP tasks and teaching you how to apply these architectures for those tasks.

Outline

Day 1

Environment setup and data download

Fundamentals

• Introduction to supervised learning
• Introduction to computational graphs
• Introduction to NLP and NLP tasks

Representations for words: Word embeddings

• Overview of embeddings
• Hands-on exercise: Word analogy problems

Deep learning frameworks

• Static versus dynamic
• PyTorch basics
• Hands-on exercise: PyTorch

Feed-forward networks for NLP

• Multilayer perceptrons
• Hands-on exercise: Chinese document classification

Modeling subword units with convolutional networks

• *Hands-on exercise: Classifying names to ethnicities

Day 2

Sequence modeling

• Basics of modeling sequences
• Representing sequences as tensors

Recurrent neural networks (RNNs) to model sequences

• Basic concepts
• Hands-on exercise: Language modeling using a character RNN
• Gated variants (LSTM and GRU)
• Structural variants (bidirectional, stacked, and tree)

Structured prediction methods

• Greedy selection
• Monte Carlo methods
• Beam search
• Viterbi decoding
• Hands-on exercise: Generating sentences from a character RNN

Attention

• Basic concepts
• Applications: Context-aware modeling

From sequence models to sequence-to-sequence models

• Basic ideas
• Applications: Translation and summarization

Advanced topics

• Memory networks
• Convolutions for sequence modeling
• Transfer learning
• Multitask learning

DL modeling for common NLP tasks

• Language modeling
• POS tagging
• Chunking
• NER
• Parsing
• Machine translation
• Summarization
• Generation

Choose your own adventure

• Hands-on exercise: Work with an NLP problem end-to-end from a selection of problems

DL for NLP: Best practices

Wrap-up and Q&A

• When to use deep learning for NLP
• When not to use deep learning for NLP