Conv-MPN: Convolutional Message Passing Neural Network for Structured Outdoor Architecture Reconstruction

By Fuyang Zhang, Nelson Nauata, Yasutaka Furukawa. 2019

68 citations recorded by the source corpus.

This paper proposes a novel message passing neural (MPN) architecture Conv-MPN, which reconstructs an outdoor building as a planar graph from a single RGB image. Conv-MPN is specifically designed for cases where nodes of a graph have explicit spatial embedding. In our problem, nodes correspond to building edges in an image. Conv-MPN is different from MPN in that 1) the feature associated with a node is represented as a feature volume instead of a 1D vector; and 2) convolutions encode messages instead of fully connected layers. Conv-MPN learns to select a true subset of nodes (i.e., building edges) to reconstruct a building planar graph. Our qualitative and quantitative evaluations over 2,000 buildings show that Conv-MPN makes significant improvements over the existing fully neural solutions. We believe that the paper has a potential to open a new line of graph neural network research for structured geometry reconstruction.

Subfields: Computer Vision and Pattern Recognition, Environmental Engineering, Graph Learning, 3D Vision & Pose

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Environmental Engineering

In Graph Learning

In 3D Vision & Pose

Citations in the corpus

House-GAN: Relational Generative Adversarial Networks for Graph-constrained House Layout Generation (2020, methodology): “In particular, we employ Conv-MPN [26], which diﬀers from GCNs [3,12] in that a node stores a feature volume and convolutions update features in the design space (as opposed to a 1D latent vector space).”