Natural language understanding at scale with Spark NLP

Description

NLP is a key component in many data science systems that must understand or reason about text. Common use cases include question answering, entity recognition, sentiment analysis, dependency parsing, de-identification, and natural language BI. Building such systems usually requires combining three types of software libraries: NLP annotation frameworks, machine learning frameworks, and deep learning frameworks.

David Talby, Alex Thomas, Saif Addin Ellafi, and Claudiu Branzan walk you through state-of-the-art natural language processing (NLP) using the highly performant, highly scalable open source Spark NLP library. You’ll spend about half your time coding as you work through four sections, each with an end-to-end working codebase that you can change and improve.

Outline:

Using Spark NLP to build an NLP pipeline that can understand text structure, grammar, and sentiment and perform entity recognition

• When is an NLP library needed?
• Introduction to Spark NLP
• Benchmarks and scalability
• Built-in Spark NLP annotators
• Core NLP tasks: Tokenizer, normalizer, stemmer, lemmatizer, chunker, POS, and named-entity recognition (NER)
• Using pretrained models and pipelines

Building a machine learning pipeline that includes and depends on NLP annotators to generate features

• Feature engineering and optimization
• Trainable NLP tasks: Spell checker, sentiment analysis, NER
• Applying word embeddings to “featurize” text
• Best practices and common pitfalls for creating unified NLP and ML pipelines

Using Spark NLP with TensorFlow to train deep learning models for state-of-the-art NLP

• Why you’ll need to train domain-specific NLP models for most real-world use cases
• Recent deep learning research results for named entity recognition, entity resolution, assertion status detection, and de-identification
• Spark NLP and TensorFlow integration and benefits
• Training your own domain-specific deep learning NLP models
• Best practices for choosing between alternative NLP algorithms and annotators

Advanced Spark NLP functionality that enables a scalable open source solution to more complex language-understanding use cases

• Optical character recognition (OCR) annotators and pipelines
• Improving OCR accuracy with customized dictionaries, forms, and spell checkers
• Entity resolution versus named entity recognition
• An overview of state-of-the-art NLP algorithms and models for healthcare