GraphGPT: Graph Instruction Tuning for Large Language Models

By Jiabin Tang, Yuhao Yang, Wei Wei, Lei Shi, Lixin Su, Suqi Cheng, Dawei Yin, Chao Huang. Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 2023

364 citations recorded by the source corpus.

Graph Neural Networks (GNNs) have evolved to understand graph structures through recursive exchanges and aggregations among nodes. To enhance robustness, self-supervised learning (SSL) has become a vital tool for data augmentation. Traditional methods often depend on fine-tuning with task-specific labels, limiting their effectiveness when labeled data is scarce. Our research tackles this by advancing graph model generalization in zero-shot learning environments. Inspired by the success of large language models (LLMs), we aim to create a graph-oriented LLM capable of exceptional generalization across various datasets and tasks without relying on downstream graph data. We introduce the GraphGPT framework, which integrates LLMs with graph structural knowledge through graph instruction tuning. This framework includes a text-graph grounding component to link textual and graph structures and a dual-stage instruction tuning approach with a lightweight graph-text alignment projector. These innovations allow LLMs to comprehend complex graph structures and enhance adaptability across diverse datasets and tasks. Our framework demonstrates superior generalization in both supervised and zero-shot graph learning tasks, surpassing existing benchmarks. The open-sourced model implementation of our GraphGPT is available at https://github.com/HKUDS/GraphGPT.

Subfields: Artificial Intelligence

View Paper: https://www.semanticscholar.org/paper/45872b94798c3125abfb185b7926689c5e767763

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Jiliang Tang
2. Heng Ji
3. Jiawei Han
4. Dawei Yin
5. Meng Jiang
6. Hui Liu
7. Wenqi Fan
8. Wei Jin

Citations in the corpus

• LightRAG: Simple and Fast Retrieval-Augmented Generation (2024, neutral): “Furthermore, LightRAG’s seamless incremental update capability ensures that the system remains current and responsive to new information, thereby maintaining its effectiveness over time.”
• Exploring the Potential of Large Language Models (LLMs)in Learning on Graphs (2023, background): “However, we observe that the probability of the outputs from these models is consistently close to 1, rendering the output not helpful.”
• Large Language Models on Graphs: A Comprehensive Survey (2023, methodology): “Instruction tuning is adopted in the graph domain for node classification [45], link prediction [46], and graph-level tasks [47].”
• G-Retriever: Retrieval-Augmented Generation for Textual Graph Understanding and Question Answering (2024, neutral): “This exploration spans diverse aspects, ranging from the design of general graph models [47, 25, 51, 19, 40, 31], and multi-modal architectures [23, 49] to practical applications.”
• Griffin: Towards a Graph-Centric Relational Database Foundation Model (2025, unknown): “UniTabE (Yang et al., 2024) and TPBerta (Yan et al., 2024)”