14–17 Oct 2019

Industrialized capsule networks for text analytics

Abhishek Kumar (Publicis Sapient)
14:3515:15 Wednesday, 16 October 2019
Location: Buckingham Room - Palace Suite

Who is this presentation for?

  • Data scientists, data engineers, data architects, CxOs, and software engineers




Multilabel text classification is an interesting problem where multiple tags or categories may have to be associated with text or documents. Multilabel text classification occurs in numerous real-world scenarios, for instance, in news categorization and in bioinformatics. The Kaggle dataset is representative of the problem.

Several other interesting problems in text analytics exist, such as abstractive summarization, sentiment analysis, search and information retrieval, entity resolution, document categorization, document clustering, machine translation, etc. Many have attempted to solve the problems by applying deep learning—for instance, an early approach to applying a convolutional network to make effective use of word order in text categorization. Recurrent neural networks (RNNs) have been effective in various tasks in text analytics, and have achieved significant progress in language translation by modeling machine translation using an encoder-decoder approach with the encoder formed by a neural network.

However, certain cases require modeling the hierarchical relationship in text data, which is difficult to achieve with traditional deep learning networks because linguistic knowledge may have to be incorporated in these networks to achieve high accuracy. Moreover, deep learning networks don’t consider hierarchical relationships between local features, as the pooling operation of CNNs loses information about the hierarchical relationships.

Abhishek Kumar explores an industrial-scale use case of capsule networks implemented for a client in the realm of text analytics—news categorization. He shows, using precision, recall, and F1 metrics, the performance of capsule networks on the news categorization task. He also benchmarks the performance of recurrent capsule networks for the same task and compares the two implementations against a baseline model. Importantly, you’ll learn how to tune the hyperparameters of capsule networks, such as batch size, number and size of filters, initial learning rate, and the number and dimension of capsules. Abhishek also shares some key challenges he faced.

Prerequisite knowledge

  • A basic understanding of NLP and deep learning

What you'll learn

  • Discover the motivation for capsule networks and how they can be used in text analytics
  • Get an overview of recurrent capsule networks
  • Learn about the implementation RCNs in TensorFlow and PyTorch
  • See the benchmarking of capsule networks with dynamic routing and recurrent capsule networks for a real multilabel text classification use case for news categorization
Photo of Abhishek Kumar

Abhishek Kumar

Publicis Sapient

Abhishek Kumar is a senior manager of data science in Publicis Sapient’s India office, where he looks after scaling up the data science practice by applying machine learning and deep learning techniques to domains such as retail, ecommerce, marketing, and operations. Abhishek is an experienced data science professional and technical team lead specializing in building and managing data products from conceptualization to the deployment phase and interested in solving challenging machine learning problems. Previously, he worked in the R&D center for the largest power-generation company in India on various machine learning projects involving predictive modeling, forecasting, optimization, and anomaly detection and led the center’s data science team in the development and deployment of data science-related projects in several thermal and solar power plant sites. Abhishek is a technical writer and blogger as well as a Pluralsight author and has created several data science courses. He’s also a regular speaker at various national and international conferences and universities. Abhishek holds a master’s degree in information and data science from the University of California, Berkeley. Abhishek has spoken at past O’Reilly conferences, including Strata 2019, Strata 2018, and AI 2019.

Contact us


For conference registration information and customer service


For more information on community discounts and trade opportunities with O’Reilly conferences


For information on exhibiting or sponsoring a conference


For media/analyst press inquires