Language is All a Graph Needs

By Ruosong Ye, Caiqi Zhang, Runhui Wang, Shuyuan Xu, Yongfeng Zhang. Findings, 2023

307 citations recorded by the source corpus.

The emergence of large-scale pre-trained language models has revolutionized various AI research domains. Transformers-based Large Language Models (LLMs) have gradually replaced CNNs and RNNs to unify fields of computer vision and natural language processing. Compared with independent data like images, videos or texts, graphs usually contain rich structural and relational information. Meanwhile, languages, especially natural language, being one of the most expressive mediums, excels in describing complex structures. However, existing work on incorporating graph problems into the generative language modeling framework remains very limited. Considering the rising prominence of LLMs, it becomes essential to explore whether LLMs can also replace GNNs as the foundation model for graphs. In this paper, we propose InstructGLM (Instruction-finetuned Graph Language Model) with highly scalable prompts based on natural language instructions. We use natural language to describe multi-scale geometric structure of the graph and then instruction finetune an LLM to perform graph tasks, which enables Generative Graph Learning. Our method surpasses all GNN baselines on ogbn-arxiv, Cora and PubMed datasets, underscoring its effectiveness and sheds light on generative LLMs as new foundation model for graph machine learning. Our code is available at https://github.com/agiresearch/InstructGLM.

Subfields: Artificial Intelligence, Cultural Studies, Molecular Biology

View Paper: https://www.semanticscholar.org/paper/8d65b5940a4dbed8c18e02ca35e3a9d7a14ea76b

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Jiliang Tang
2. Heng Ji
3. Jiawei Han
4. Dawei Yin
5. Meng Jiang
6. Hui Liu
7. Wenqi Fan
8. Wei Jin

In Cultural Studies

1. Jiawei Han

In Molecular Biology

1. Heng Ji
2. Wei Jin
3. Meng Jiang
4. Jiawei Han
5. Jiliang Tang
6. Bowen Jin
7. Chi Han
8. Gang Liu

Citations in the corpus

• OpenAGI: When LLM Meets Domain Experts (2023, background): “This results in superior performance across diverse natural language processing tasks [2, 36].”
• Exploring the Potential of Large Language Models (LLMs)in Learning on Graphs (2023, background): “However, we observe that the probability of the outputs from these models is consistently close to 1, rendering the output not helpful.”
• Large Language Models on Graphs: A Comprehensive Survey (2023, methodology): “Instruction tuning is adopted in the graph domain for node classification [45], link prediction [46], and graph-level tasks [47].”
• Graph Retrieval-Augmented Generation: A Survey (2024, unknown): “We illustrate this process with an example in Figure 5, with detailed introductions provided below.”
• G-Retriever: Retrieval-Augmented Generation for Textual Graph Understanding and Question Answering (2024, unknown): “This exploration spans diverse aspects, ranging from the design of general graph models [47, 25, 51, 19, 40, 31], and multi-modal architectures [23, 49] to practical applications.”