Auto-GNN: Neural architecture search of graph neural networks

By Kaixiong Zhou, Qingquan Song, Xiao Huang, Xia Hu. 2019

217 citations recorded by the source corpus.

Graph neural networks (GNNs) have been widely used in various graph analysis tasks. As the graph characteristics vary significantly in real-world systems, given a specific scenario, the architecture parameters need to be tuned carefully to identify a suitable GNN. Neural architecture search (NAS) has shown its potential in discovering the effective architectures for the learning tasks in image and language modeling. However, the existing NAS algorithms cannot be applied efficiently to GNN search problem because of two facts. First, the large-step exploration in the traditional controller fails to learn the sensitive performance variations with slight architecture modifications in GNNs. Second, the search space is composed of heterogeneous GNNs, which prevents the direct adoption of parameter sharing among them to accelerate the search progress. To tackle the challenges, we propose an automated graph neural networks (AGNN) framework, which aims to find the optimal GNN architecture efficiently. Specifically, a reinforced conservative controller is designed to explore the architecture space with small steps. To accelerate the validation, a novel constrained parameter sharing strategy is presented to regularize the weight transferring among GNNs. It avoids training from scratch and saves the computation time. Experimental results on the benchmark datasets demonstrate that the architecture identified by AGNN achieves the best performance and search efficiency, comparing with existing human-invented models and the traditional search methods.

Subfields: Artificial Intelligence, Neurology, Graph Learning, Neural Architecture Search & AutoML

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Xia Hu
2. Quanming Yao
3. Qingquan Song
4. Irwin King
5. Tong Zhang
6. Yi-Wei Chen
7. Yaoman Li
8. Huan Zhao

In Neurology

1. Quanming Yao
2. Irwin King
3. Yaoman Li
4. Xia Hu
5. Qingming Huang
6. Pietro Lio'
7. Tong Zhang

In Graph Learning

1. Quanming Yao
2. Irwin King
3. Yaoman Li
4. Xia Hu
5. Rui Chen
6. Min Shi
7. Qingming Huang
8. Xingquan Zhu

In Neural Architecture Search & AutoML

1. Quanming Yao
2. Xia Hu
3. Yi-Wei Chen
4. Qingquan Song
5. Irwin King
6. Yaoman Li
7. Tong Zhang
8. Kaixiong Zhou

Citations in the corpus

• AutoML: A Survey of the State-of-the-Art (2019, methodology): “Therefore, NAS has also been applied to many other computer vision tasks, such as object detection [153, 154, 155], semantic segmentation [156, 157], image restoration [158], GAN [159], graph neural network (GNN) [160].”
• Probabilistic Dual Network Architecture Search on Graphs (2020, methodology): “In Table 2, we present the results of PDNAS for this setting in comparison to graph attention networks (GAT) (Veliˇckovi´c et al., 2018), GraphNAS (Gao et al., 2019) and AGNN (Zhou et al., 2019).”
• Simplifying Architecture Search for Graph Neural Network (2020, methodology): “To summarize, the contributions of this work are in the following: • In this work, to automatically obtain well-behaved data-specific GNN architectures, we propose the SNAG framework, which can overcome the drawbacks of existing NAS approaches, i.e., GraphNAS and Auto-GNN.”
• Towards Deeper Graph Neural Networks with Differentiable Group Normalization (2020, methodology): “Graph neural networks (GNNs) [1, 2, 3] have emerged as a promising tool for analyzing networked data, such as biochemical networks [4, 5], social networks [6, 7], and academic networks [8, 9].”
• Propagation Model Search for Graph Neural Networks (2020, methodology): “We also compare against AGNN (Zhou et al. 2019) with parameter sharing which performs better than GraphNAS (Gao et al. 2019).”
• Pooling Architecture Search for Graph Classification (2021, methodology): “For example, GraphNAS [11], AutoGNN [57], AutoGM [48], DSS [25] and [30] learn to design aggregation layers with diverse dimensions, such as attention function, attention head number, embedding size, etc; SANE [56], SNAG [55] and AutoGraph [23] provide the extra skip connections learning; GNAS [3] a”
• Evolutionary Architecture Search for Graph Neural Networks (2020, methodology): “Auto-GNN [16] follows a similar architecture searching paradigm as GraphNAS while proposing a parameter sharing strategy that enables homogeneous architectures to share parameters.”
• AutoGraph: Automated Graph Neural Network (2020, methodology): “RNN architectures for image and language modeling tasks, the GNN is very different from CNN or RNN. Thus they cannot be directly applied to the GNN architecture search. Gao et al. [6] and Zhou et al. [25] propose a new schema to encode the GNN architecture and apply reinforcement learning to search ”
• Rethinking Graph Neural Architecture Search from Message-passing (2021, background): “Recently, researchers [28, 41] use existing GNNs (GCN [12], GAT [33], etc.) and hyper parameters (the head number of GAT, etc.) as atomic operations for searching.”
• Techniques for Automated Machine Learning (2019, methodology): “Recently, AutoDL has been developed to deal with segmentation [58; 91], graph-CNN [29; 100], GAN [33], and autoencoders [80].”