Large Language Models on Graphs: A Comprehensive Survey
By Bowen Jin, Gang Liu, Chi Han, Meng Jiang, Heng Ji, Jiawei Han. IEEE Transactions on Knowledge and Data Engineering, 2023
322 citations recorded by the source corpus.
Large language models (LLMs), such as GPT4 and LLaMA, are creating significant advancements in natural language processing, due to their strong text encoding/decoding ability and newly found emergent capability (e.g., reasoning). While LLMs are mainly designed to process pure texts, there are many real-world scenarios where text data is associated with rich structure information in the form of graphs (e.g., academic networks, and e-commerce networks) or scenarios where graph data is paired with rich textual information (e.g., molecules with descriptions). Besides, although LLMs have shown their pure text-based reasoning ability, it is underexplored whether such ability can be generalized to graphs (i.e., graph-based reasoning). In this paper, we provide a systematic review of scenarios and techniques related to large language models on graphs. We first summarize potential scenarios of adopting LLMs on graphs into three categories, namely pure graphs, text-attributed graphs, and text-paired graphs. We then discuss detailed techniques for utilizing LLMs on graphs, including LLM as Predictor, LLM as Encoder, and LLM as Aligner, and compare the advantages and disadvantages of different schools of models. Furthermore, we discuss the real-world applications of such methods and summarize open-source codes and benchmark datasets. Finally, we conclude with potential future research directions in this fast-growing field.
Subfields: Artificial Intelligence
View Paper: https://www.semanticscholar.org/paper/67239d6e9c2c5f8a6d19cb35154e5aa7eaa00f51
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
Citations in the corpus
- Knowledge Graph Prompting for Multi-Document Question Answering (2023, background): “Pre-train, Prompt, and Predict with LLMs With the emergence of LLMs, the paradigm of ‘pre-train, prompt, predict’ has gained magnificent popularity in handling a wide spectrum of tasks (Gururangan et al. 2020; Liu et al. 2023; Yu et al. 2023).”
- Graph Retrieval-Augmented Generation: A Survey (2024, unknown): “Although LLMs are primarily designed to process pure text and struggle with non-Euclidean data containing complex structural information, such as graphs [41, 153], numerous studies [13, 28, 65, 83, 92, 105, 119, 120, 161, 189] have been conducted in these fields.”
- G-Retriever: Retrieval-Augmented Generation for Textual Graph Understanding and Question Answering (2024, unknown): “A significant body of research has emerged at the intersection of graph-based techniques and LLMs [30, 24, 15, 44, 54].”
- SimGRAG: Leveraging Similar Subgraphs for Knowledge Graphs Driven Retrieval-Augmented Generation (2024, unknown): “Recently, the pre-trained large language models have shown the ability to understand and handle knowledge graph (KG) related tasks (Pan et al., 2023; Jin et al., 2024; Pan et al., 2024; Yang et al., 2024; Li et al., 2024b), such as KG construction (Zhu et al., 2024b), KG completion (Xie et al.,…”