D-VAE: A Variational Autoencoder for Directed Acyclic Graphs

By Muhan Zhang, Shali Jiang, Zhicheng Cui, Roman Garnett, Yixin Chen. 2019

238 citations recorded by the source corpus.

Graph structured data are abundant in the real world. Among different graph types, directed acyclic graphs (DAGs) are of particular interest to machine learning researchers, as many machine learning models are realized as computations on DAGs, including neural networks and Bayesian networks. In this paper, we study deep generative models for DAGs, and propose a novel DAG variational autoencoder (D-VAE). To encode DAGs into the latent space, we leverage graph neural networks. We propose an asynchronous message passing scheme that allows encoding the computations on DAGs, rather than using existing simultaneous message passing schemes to encode local graph structures. We demonstrate the effectiveness of our proposed DVAE through two tasks: neural architecture search and Bayesian network structure learning. Experiments show that our model not only generates novel and valid DAGs, but also produces a smooth latent space that facilitates searching for DAGs with better performance through Bayesian optimization.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

BANANAS: Bayesian Optimization with Neural Architectures for Neural Architecture Search (2019, methodology): “[75] Muhan Zhang, Shali Jiang, Zhicheng Cui, Roman Garnett, and Yixin Chen.”
On the Binding Problem in Artificial Neural Networks (2020, methodology): “Recently, MPNNs were extended to allow for continuous updates (Deng et al., 2019; Liu et al., 2019). then optimize for the right structure in the corresponding space, e.g. using VAEs (Kusner et al., 2017; Zhang et al., 2019), or GANs (Yang et al., 2019).”
Automated Machine Learning: From Principles to Practices (2018, methodology): “There exist works in AutoML area [2, 110, 152] that make use of the idea of stochastic relaxation for search space transformation to handle the search problem.”
NPENAS: Neural Predictor Guided Evolution for Neural Architecture Search (2022, methodology): “The encoding methods roughly fall into two categories—vector encoding [3], [6]–[8], [14] and graph encoding [2], [13], [15], [16].”
Does Unsupervised Architecture Representation Learning Help Neural Architecture Search? (2020, methodology): “[32] proposes an asynchronous message passing scheme to encode DAG computations using RNNs.”
Directed Acyclic Graph Network for Conversational Emotion Recognition (2021, background): “ConGCN (Zhang et al., 2019a) regards both speakers and utterances as graph nodes and makes the whole ERC dataset a single graph.”
Sample-Efficient Optimization in the Latent Space of Deep Generative Models via Weighted Retraining (2020, background): “automatic machine learning [41, 42, 76], conditional image generation [48, 47], and model explainability [2].”
A Study on Encodings for Neural Architecture Search (2020, methodology): “We also train a neural predictor model using seven different encodings, as well as GCN [21] and VAE [25].”
DiBS: Differentiable Bayesian Structure Learning (2021, background): “Follow-up work has leveraged this insight to model nonlinear relationships [24–27], time-series data [28], in the context of generative modeling [29, 30], for causal inference [31–33], and contributed to its theoretical understanding [34, 35].”