Put AI to work
June 26-27, 2017: Training
June 27-29, 2017: Tutorials & Conference
New York, NY

Deep learning in the fashion industry

Pau Carré (Gilt)
4:00pm4:40pm Thursday, June 29, 2017
Implementing AI
Location: Murray Hill E/W Level: Intermediate
Secondary topics:  Deep Learning, Fashion, Retail and e-commerce, Vision

Prerequisite Knowledge

  • Basic experience training deep neural networks
  • An understanding of the convolutional operator

What you'll learn

  • Learn how Gilt uses deep learning to automatically detect similar products and identify facets in dresses


In the fashion industry, many tasks require human-level cognitive skills, such as detecting similar products or identifying facets in products like sleeve length or silhouette types in dresses. Pau Carré explains how Gilt is reshaping the fashion industry by leveraging the power of deep learning and GPUs to address these challenges.

Gilt is building automated faceting systems to detect dresses based on their silhouette, neckline, sleeve type and occasion. On top of that, it is also developing systems to detect dress similarity, which can be useful for product recommendations. Furthermore, when integrated with automated faceting, customers will be able to find similar products with different facets. For instance, a customer might be very interested in a particular dress but wants a different neckline or sleeve length.

Topics include:

  • Dress faceting: Gilt uses image classification for dress faceting, leveraging Facebook’s open source Torch implementation of Microsoft’s ResNet. This approach focuses on generating a high-quality data-set using a feedback loop. The initial dataset is generated from a database query or scraping websites. Then a neural network is trained and used to identify the most likely mistagged images in the dataset. They are then retagged using fashion experts and Amazon Mechanical Turk. The network is then retrained with the corrected dataset.

  • Dress similarity: Gilt uses TiefVision, a project it has open-sourced, which uses transfer learning to reuse an existing already trained network for another task. Gilt transfers the network into new networks by replacing its last layers with new ones specialized for other purposes. The two new networks are trained to locate the dress in the image following the process outlined in Yann LeCun’s “OverFeat” paper. One network detects whether an image patch belongs to a dress or to the background; the other is trained to detect the dress given an image patch. Combining both networks, it’s possible to extract an accurate location of the dress. Then another network is trained for supervised similarity using deep ranking. The supervised learning network uses three images as input: a reference dress, a dress similar to the reference, and another dissimilar to the reference. Using a Siamese network and training the network using a hinge loss function, the network learns to detect dress similarities.
Photo of Pau Carré

Pau Carré


Pau Carré is a deep learning software engineer at Gilt. Pau has 10 years of experience encompassing software security, IT management, microwave networks profiling, quality engineering, and deep learning and functional programming for the fashion industry. Over his career, he has lived and worked in cosmopolitan Barcelona, paradisiac Mallorca, and magnificent Vienna, and is now based in welcoming Dublin.

Leave a Comment or Question

Help us make this conference the best it can be for you. Have questions you'd like this speaker to address? Suggestions for issues that deserve extra attention? Feedback that you'd like to share with the speaker and other attendees?

Join the conversation here (requires login)