Explainable Outfit Recommendation with Joint Outfit Matching and Comment Generation

By Yujie Lin, Pengjie Ren, Zhumin Chen, Zhaochun Ren, Jun Ma, Maarten de Rijke. 2018

118 citations recorded by the source corpus.

Most previous work on outfit recommendation focuses on designing visual features to enhance recommendations. Existing work neglects user comments of fashion items, which have been proven to be effective in generating explanations along with better recommendation results. We propose a novel neural network framework, neural outfit recommendation (NOR), that simultaneously provides outfit recommendations and generates abstractive comments. Neural outfit recommendation (NOR) consists of two parts: outfit matching and comment generation. For outfit matching, we propose a convolutional neural network with a mutual attention mechanism to extract visual features. The visual features are then decoded into a rating score for the matching prediction. For abstractive comment generation, we propose a gated recurrent neural network with a cross-modality attention mechanism to transform visual features into a concise sentence. The two parts are jointly trained based on a multi-task learning framework in an end-to-end back-propagation paradigm. Extensive experiments conducted on an existing dataset and a collected real-world dataset show NOR achieves significant improvements over state-of-the-art baselines for outfit recommendation. Meanwhile, our generated comments achieve impressive ROUGE and BLEU scores in comparison to human-written comments. The generated comments can be regarded as explanations for the recommendation results. We release the dataset and code to facilitate future research.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Xu Chen
2. Yongfeng Zhang
3. Zheng Qin
4. Hanxiong Chen
5. Yixin Cao
6. Hongyuan Zha
7. Hongteng Xu
8. Xiangnan He

In Computer Vision and Pattern Recognition

1. Yongfeng Zhang
2. Xu Chen
3. Zheng Qin
4. Hanxiong Chen
5. Yixin Cao
6. Hongyuan Zha
7. Hongteng Xu
8. Liqiang Nie

Citations in the corpus

• Neural Compatibility Modeling With Probabilistic Knowledge Distillation (2020, methodology): “1) Dataset: In this work, to evaluate our model, we adopted two real-world datasets FashionVC [30] and ExpFashion [9],”
• Personalized Fashion Recommendation with Visual Explanations based on Multimodal Attention Network: Towards Visually Explainable Recommendation (2019, background): “On the other hand, there are also many models [21, 22, 28, 35] exploring to provide explanation in a "generative (or abstractive)" manner.”