Presented By
O’Reilly + Cloudera
Make Data Work
March 25-28, 2019
San Francisco, CA

NLP from scratch: Solving the cold start problem for natural language processing

Michael Johnson (Lockheed Martin), Norris Heintzelman (Lockheed Martin)
11:50am12:30pm Wednesday, March 27, 2019
Average rating: ****.
(4.60, 15 ratings)

Who is this presentation for?

  • Machine learning practitioners, data scientists, and business analysts



Prerequisite knowledge

  • Familiarity with machine learning and NLP (useful but not required)

What you'll learn

  • Explore cutting-edge techniques for building machine learning models from scratch in the NLP domain
  • Understand how to frame a business problem as an NLP problem
  • Learn strategies for getting a solution off the ground
  • Discover mistakes made in the process and how to avoid them


Unstructured data in the form of documents, web pages, and social media interactions is an ever-growing, ever-more valuable data source for addressing present business problems, from exploring brand sentiment to identifying sensitive information in internal documents. Unfortunately, the classification and annotation algorithms behind solving these problems often require significant amounts of labeled training data to produce desired accuracy.

Michael Johnson and Norris Heintzelman share several techniques they’ve implemented to build classification and NER models from scratch. They lead a tour through this space as it applies to NLP and demonstrate their approach and architecture for the following techniques:

  • Weak supervision for news documents: Using rules base classification alongside deep learning system for text classification
  • Active learning and human in the loop: Explaining how breakthroughs in transfer learning for NLP have impacted their active learning framework for building an LSTM-based relevance model
  • Creative training sets: Identifying and cleaning already-labeled datasets, training classifier on “only” positive examples
  • NER adjudication: Combining knowledge from several annotation sources that leverages the strengths of each source

For each of these topics, Michael and Norris outline the theoretical foundation, the implementation architecture, and tools used and discuss the problems they encountered—so you can avoid making the same mistakes.

Photo of Michael Johnson

Michael Johnson

Lockheed Martin

Michael Johnson is a senior data scientist at Lockheed Martin. He has done data science and analytics in fields including manufacturing optimization, semiconductor reliability, and human resources-focused time series forecasting and simulation. He has recently been focused on how to apply cutting-edge deep learning algorithms to NLP domains.

Photo of Norris Heintzelman

Norris Heintzelman

Lockheed Martin

Norris Heintzelman is a senior research and data scientist with 19 years’ real-world experience converting data into knowledge—that is, 19 years’ experience in many areas of natural language processing, knowledge systems, cleaning and normalizing messy data, and rigorous accuracy measurement. Norris has published several papers in the fields of health informatics and general knowledge management. She has worked for Lockheed Martin for a very long time, in multiple business areas, from public sector contracts to advanced R&D to internal business process support. An alumna of both Temple University and the University of Pennsylvania, she lives in Wilmington, Delaware, with her husband, two daughters, and two cats. She likes to eat and talk about food.