Exploring the Potential of Large Language Models (LLMs)in Learning on Graphs
By Zhikai Chen, Haitao Mao, Hang Li, Wei Jin, Haifang Wen, Xiaochi Wei, Shuaiqiang Wang, Dawei Yin, Wenqi Fan, Hui Liu and 1 more. SIGKDD Explorations, 2023
443 citations recorded by the source corpus.
Learning on Graphs has attracted immense attention due to its wide real-world applications. The most popular pipeline for learning on graphs with textual node attributes primarily relies on Graph Neural Networks (GNNs), and utilizes shallow text embedding as initial node representations, which has limitations in general knowledge and profound semantic understanding. In recent years, Large Language Models (LLMs) have been proven to possess extensive common knowledge and powerful semantic comprehension abilities that have revolutionized existing workflows to handle text data. In this paper, we aim to explore the potential of LLMs in graph machine learning, especially the node classification task, and investigate two possible pipelines: LLMs-as-Enhancers and LLMs-as-Predictors. The former leverages LLMs to enhance nodes' text attributes with their massive knowledge and then generate predictions through GNNs. The latter attempts to directly employ LLMs as standalone predictors. We conduct comprehensive and systematical studies on these two pipelines under various settings. From comprehensive empirical results, we make original observations and find new insights that open new possibilities and suggest promising directions to leverage LLMs for learning on graphs. Our codes and datasets are available at: https://github.com/CurryTang/Graph-LLM .
Subfields: Artificial Intelligence
View Paper: https://www.semanticscholar.org/paper/105669ec59a58fb2d4dd3021a984af33c227c5ab
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
Citations in the corpus
- Large Language Models for Education: A survey and outlook (2024, unknown): “Furthermore, advancements in AI techniques have introduced new analytical tools and insights that continue to enhance pedagogical practices in education.”
- Recommender Systems in the Era of Large Language Models (LLMs) (2023, unknown): “......”
- Language is All a Graph Needs (2023, background): “Wang et al. (2023a) and Chen et al. (2023b) both explore using natural language with LLM for graph problems, with (Wang et al., 2023a) focusing on mathematical problems on small graphs while (Chen et al., 2023b) concentrating on node classification in Text-Attributed Graphs (TAGs) (Hu et al., 2020).”
- Large Language Models on Graphs: A Comprehensive Survey (2023, background): “Recently, there has been an increasing interest [9] in extending LLMs for graph-based applications (summarized in Fig.”
- Can Language Models Solve Graph Problems in Natural Language? (2023, background): “Recently, Chen et al. [2023] explore the potential of LLMs on the graph node classification task.”
- G-Retriever: Retrieval-Augmented Generation for Textual Graph Understanding and Question Answering (2024, unknown): “Noteworthy applications include fundamental graph reasoning [52, 3, 56], node classification [8, 11, 39, 5, 50, 4, 33], graph classification/regression [32, 55], and leveraging LLMs for knowledge graph-related tasks [41, 14, 29].”
- StructGPT: A General Framework for Large Language Model to Reason over Structured Data (2023, background): “Benefitting from the strong few-shot and zero-shot capability, recent studies have leveraged LLMs to perform reasoning over structured data (Chen et al., 2023; Li et al., 2023a; Cheng et al., 2022; Rajkumar et al., 2022).”
- GraphGPT: Graph Instruction Tuning for Large Language Models (2023, unknown): “This significant reduction in token consumption translates to a substantial decrease in training and inference resource requirements.”