Neural News Recommendation with Multi-Head Self-Attention

By Chuhan Wu, Fangzhao Wu, Suyu Ge, Tao Qi, Yongfeng Huang, Xing Xie. Conference on Empirical Methods in Natural Language Processing, 2019

366 citations recorded by the source corpus.

News recommendation can help users find interested news and alleviate information overload. Precisely modeling news and users is critical for news recommendation, and capturing the contexts of words and news is important to learn news and user representations. In this paper, we propose a neural news recommendation approach with multi-head self-attention (NRMS). The core of our approach is a news encoder and a user encoder. In the news encoder, we use multi-head self-attentions to learn news representations from news titles by modeling the interactions between words. In the user encoder, we learn representations of users from their browsed news and use multi-head self-attention to capture the relatedness between the news. Besides, we apply additive attention to learn more informative news and user representations by selecting important words and news. Experiments on a real-world dataset validate the effectiveness and efficiency of our approach.

Subfields: Artificial Intelligence, Information Systems

View Paper: https://www.semanticscholar.org/paper/508b55bed9187adabdd1db7c7bd0bca4b1c221bf

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Xing Xie
2. Fangzhao Wu
3. Chuhan Wu
4. Yongfeng Huang
5. Yongfeng Zhang
6. Tao Qi
7. M. Zhou
8. Jianfeng Gao

In Information Systems

1. Xing Xie
2. Fangzhao Wu
3. Chuhan Wu
4. Yongfeng Huang
5. Tao Qi
6. Yongfeng Zhang
7. Zuohui Fu
8. Yingqiang Ge

Citations in the corpus

• Recommendation as Language Processing (RLP): A Unified Pretrain, Personalized Prompt & Predict Paradigm (P5) (2022, background): “The main work mostly address four lines of research: 1) explainable recommendation [11, 16, 28–30, 71, 73] where NLP models help generating text explanations for a given recommendation; 2) sequential recommendation as language modeling [10, 56, 76] which considers user interaction histories as word ”
• FedRec: Privacy-Preserving News Recommendation with Federated Learning (2020, methodology): “…attentive multi-view learning; (7) NPA (Wu et al., 2019b), using personalized attention network to learn news and user representations; (8) NRMS (Wu et al., 2019d), learning representations of news and users via multi-head self-attention network; (9) FCF (Ammad et al., 2019), a federated…”
• MIND: A Large-scale Dataset for News Recommendation (2020, methodology): “First, news is a common form of texts, and text modeling et al., 2019; Wu et al., 2019c).”
• Where to Go Next for Recommender Systems? ID- vs. Modality-based Recommender Models Revisited (2023, result): “Novelty clarification: Though much previous literature has studied MoRec and compared with many baselines [31, 41, 63, 64, 79], unfortunately none of them provided a fair or rigorous comparison between their proposed MoRec and the corresponding IDRec counterparts in regular or even warm item recomme”
• When large language models meet personalization: perspectives of challenges and opportunities (2023, positive): “NRMS [81] and CPRS [82] utilize multi-head self-attention networks to learn word representations.”
• DA-Transformer: Distance-aware Transformer (2020, methodology): “On the MIND dataset, following (Wu et al., 2019) we use the average AUC, MRR, nDCG@5 and nDCG@10 scores of all sessions as the metrics.”