Learning Fashion Compatibility with Bidirectional LSTMs

By Xintong Han, Zuxuan Wu, Yu-Gang Jiang, Larry S. Davis. ACM Multimedia, 2017

420 citations recorded by the source corpus.

The ubiquity of online fashion shopping demands effective recommendation services for customers. In this paper, we study two types of fashion recommendation: (i) suggesting an item that matches existing components in a set to form a stylish outfit (a collection of fashion items), and (ii) generating an outfit with multimodal (images/text) specifications from a user. To this end, we propose to jointly learn a visual-semantic embedding and the compatibility relationships among fashion items in an end-to-end fashion. More specifically, we consider a fashion outfit to be a sequence (usually from top to bottom and then accessories) and each item in the outfit as a time step. Given the fashion items in an outfit, we train a bidirectional LSTM (Bi-LSTM) model to sequentially predict the next item conditioned on previous ones to learn their compatibility relationships. Further, we learn a visual-semantic space by regressing image features to their semantic representations aiming to inject attribute and category information as a regularization for training the LSTM. The trained network can not only perform the aforementioned recommendations effectively but also predict the compatibility of a given outfit. We conduct extensive experiments on our newly collected Polyvore dataset, and the results provide strong qualitative and quantitative evidence that our framework outperforms alternative methods.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/1686a73010fadf9c78b2d617a74d8e981fd7a290

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Neural Compatibility Modeling With Probabilistic Knowledge Distillation (2020, methodology): “• Bi-LSTM : We chose the bidirectional LSTM model in [45] which explores the outﬁt compatibility by sequentially predicting the next item conditioned on previous ones.”
Explainable Outfit Recommendation with Joint Outfit Matching and Comment Generation (2018, background): “[12] propose to jointly learn the visual-semantic embeddings and the compatibility relation-”
Personalized Fashion Recommendation with Visual Explanations based on Multimodal Attention Network: Towards Visually Explainable Recommendation (2019, background): “[10] jointly learned a visual-semantic embedding and the”
Automatic Spatially-Aware Fashion Concept Discovery (2017, background): “There have been several clothing datasets collected recently [16, 8, 21, 6, 7].”