Transfer learning has been proven to be a tremendous success in computer vision—a result of the ImageNet competition. In the past few months, there have been several breakthroughs in natural language processing with transfer learning, namely ELMo, OpenAI Transformer, and ULMFit. Pretrained models derived from these techniques have been proven in achieving state-of-the-art results on a wide range of NLP problems. The use of pretrained models has come a long way since the introduction of word2vec and GloVe, and these two approaches are considered shallow in comparison.
David Low demonstrates how to use transfer learning on an NLP application with SOTA accuracy. David starts with an introduction to transfer learning followed by explanations on why pretrained models are handy for tackling machine learning problems with limited data as well as how they could be used as fixed feature extractor for downstream tasks and applications.
David then walks you through fine-tuning a transfer learning model to achieve state-of-the-art accuracy (92%) on a real-world sentiment classification problem using the Amazon Reviews dataset. In comparison to a FastText-based model trained on the full dataset (3.6 million samples), it takes just 1,000 samples of training data to produce a model that achieves similar performance.
David Low is the cofounder and chief data scientist at Pand.ai, a company building an AI-powered chatbot to disrupt and shape the booming conversational commerce space with deep natural language processing. He represented Singapore and the National University of Singapore (NUS) in the 2016 Data Science Games held in France, and clinched the top spot among Asian and American teams. David has been invited as a guest lecturer by NUS to conduct master classes on applied machine learning and deep learning topics. Throughout his career, David has engaged in data science projects across manufacturing, telco, ecommerce, and the insurance industry, including sales forecast modeling and influencer detection, which won him awards in several competitions and was featured on the IDA website and the NUS publication. Previously, he was a data scientist at the Infocomm Development Authority (IDA) of Singapore and was involved in research collaborations with Carnegie Mellon University (CMU) and Massachusetts Institute of Technology (MIT) on projects funded by the National Research Foundation and SMART. He competes on Kaggle and holds a top 0.2% worldwide ranking.
For exhibition and sponsorship opportunities, email strataconf@oreilly.com
For information on trade opportunities with O'Reilly conferences, email partners@oreilly.com
View a complete list of Strata Data Conference contacts
©2019, O’Reilly UK Ltd • (800) 889-8969 or (707) 827-7019 • Monday-Friday 7:30am-5pm PT • All trademarks and registered trademarks appearing on oreilly.com are the property of their respective owners. • confreg@oreilly.com