Graph Convolutional Neural Networks for Web-Scale Recommender Systems
By Rex Ying, Ruining He, Kaifeng Chen, Pong Eksombatchai, William L. Hamilton, Jure Leskovec. Knowledge Discovery and Data Mining, 2018
4,201 citations recorded by the source corpus.
Recent advancements in deep neural networks for graph-structured data have led to state-of-the-art performance on recommender system benchmarks. However, making these methods practical and scalable to web-scale recommendation tasks with billions of items and hundreds of millions of users remains an unsolved challenge. Here we describe a large-scale deep recommendation engine that we developed and deployed at Pinterest. We develop a data-efficient Graph Convolutional Network (GCN) algorithm, which combines efficient random walks and graph convolutions to generate embeddings of nodes (i.e., items) that incorporate both graph structure as well as node feature information. Compared to prior GCN approaches, we develop a novel method based on highly efficient random walks to structure the convolutions and design a novel training strategy that relies on harder-and-harder training examples to improve robustness and convergence of the model. We also develop an efficient MapReduce model inference algorithm to generate embeddings using a trained model. Overall, we can train on and embed graphs that are four orders of magnitude larger than typical GCN implementations. We show how GCN embeddings can be used to make high-quality recommendations in various settings at Pinterest, which has a massive underlying graph with 3 billion nodes representing pins and boards, and 17 billion edges. According to offline metrics, user studies, as well as A/B tests, our approach generates higher-quality recommendations than comparable deep learning based systems. To our knowledge, this is by far the largest application of deep graph embeddings to date and paves the way for a new generation of web-scale recommender systems based on graph convolutional architectures.
Subfields: Artificial Intelligence, Information Systems, Graph Learning, Recommender Systems
View Paper: https://arxiv.org/abs/1806.01973
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Yoshua Bengio
- Jure Leskovec
- Xiangnan He
- Percy Liang
- Mohit Bansal
- William L. Hamilton
- Jianfeng Gao
- Quoc V. Le
In Information Systems
In Graph Learning
- William L. Hamilton
- Jure Leskovec
- Stephan Günnemann
- Matthias Fey
- Michael M. Bronstein
- Rex Ying
- Marinka Zitnik
- Aleksandar Bojchevski
In Recommender Systems
Citations in the corpus
- Knowledge Graph Convolutional Networks for Recommender Systems (2019, methodology): “Our method also connects to PinSage [21] and GAT [15].”
- Strategies for Pre-training Graph Neural Networks (2019, methodology): “To obtain the entire graph’s representation h G , the READOUT function pools node features from the final iteration K , READOUT is a permutation-invariant function, such as averaging or a more sophisticated graph-level pooling function (Ying et al., 2018b; Zhang et al., 2018).”
- Poisoning Attacks to Graph-Based Recommender Systems (2018, background): “…designing optimized poisoning attacks to other graph-based recommender systems (e.g., graph convolutional neural network based recommender systems [36]), 3) designing optimized poisoning attacks to neural network based recommender systems (e.g., [14]), and 4) designing defenses against poisoning…”
- Position-aware Graph Neural Networks (2019, background): “…2017b) has led to advances on tasks such as node classification (Kipf & Welling, 2017), link prediction (Grover & Leskovec, 2016), graph classification (Ying et al., 2018b) and graph generation (You et al., 2018b), with successful applications across domains such as social and information networks…”
- Deep Graph Infomax (2018, methodology): “…readout to perform the best across all our experiments, we assume that its power will diminish with the increase in graph size, and in those cases, more sophisticated readout architectures such as set2vec (Vinyals et al., 2015) or DiffPool (Ying et al., 2018b) are likely to be more appropriate.”
- Weisfeiler and Leman Go Neural: Higher-order Graph Neural Networks (2018, methodology): “2017), recommender systems (Ying et al. 2018a), and the analysis of quantum interactions in molecules (Schütt et al.”
- Predict then Propagate: Graph Neural Networks meet Personalized PageRank (2018, methodology): “…by using attention mechanisms (Kearnes et al., 2016; Hamilton et al., 2017; Veličković et al., 2018), random walks (Abu-El-Haija et al., 2018a; Ying et al., 2018; Li et al., 2018), edge features (Kearnes et al., 2016; Gilmer et al., 2017; Schlichtkrull et al., 2018) and making it more…”
- Lifelong Graph Learning (2020, neutral): “[56] designed a training strategy that relied on harder-and-harder training examples to improve the robustness and convergence speed of the model.”
- Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networks (2019, background): “Graph convolutional network (GCN) [9] has become increasingly popular in addressing many graph-based applications, including semi-supervised node classification [9], link prediction [17] and recommender systems [15].”
- Simple and Deep Graph Convolutional Networks (2020, background): “, 2019), recommender systems (Ying et al., 2018), and computer vision (Zhao et al.”
- Milvus: A Purpose-Built Vector Data Management System (2021, background): “Another source of multi-vector is that many applications use more than one machine learning model even for the same object to best describe that object [30, 69].”
- Large Scale Learning on Non-Homophilous Graphs: New Benchmarks and Strong Simple Methods (2021, methodology): “A variety of scalable graph learning methods have been developed for efficient computation in larger datasets [77, 16, 75, 28, 71, 32, 20, 10].”
- Deep Graph Library: A Graph-Centric, Highly-Performant Package for Graph Neural Networks. (2019, background): “nships (edges). Its application range is broad, including molecules, social networks, knowledge graphs and recommender systems (Zitnik et al., 2018; Schlichtkrull et al., 2018; Hamilton et al., 2018; Ying et al., 2018), or in general any datasets that have structural information. As a vibrant and yo”
- A Survey on Oversmoothing in Graph Neural Networks (2023, background): “…models have been successfully applied on a variety of different tasks, e.g. in computer vision and graphics Monti et al. (2017), recommender systems Ying et al. (2018), transportation Derrow-Pinion et al. (2021), computational chemistry (Gilmer et al., 2017), drug discovery Gaudelet et al. (2021),…”
- OGB-LSC: A Large-Scale Challenge for Machine Learning on Graphs (2021, background): “Modern application domains include Webscale social networks (Ugander et al., 2011), recommender systems (Ying et al., 2018), hyperlinked Web documents (Kleinberg, 1999), knowledge graphs (KGs) (Bollacker et al., 2008; Vrandečić and Krötzsch, 2014), as well as the molecule simulation data…”
- Cross-domain Recommendation Without Sharing User-relevant Data (2019, positive): “Ying et al. [44] relied on graph conventional network for collaborative filtering to match huge mount of users and items in real recommender systems.”