Efficient Representation of Low-Dimensional Manifolds using Deep Networks

By Ronen Basri, David W. Jacobs. 2016

45 citations recorded by the source corpus.

We consider the ability of deep neural networks to represent data that lies near a low-dimensional manifold in a high-dimensional space. We show that deep networks can efficiently extract the intrinsic, low-dimensional coordinates of such data. We first show that the first two layers of a deep network can exactly embed points lying on a monotonic chain, a special type of piecewise linear manifold, mapping them to a low-dimensional Euclidean space. Remarkably, the network can do this using an almost optimal number of parameters. We also show that this network projects nearby points onto the manifold and then embeds them with little error. We then extend these results to more general manifolds.

Subfields: Computational Mechanics, Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

1. Jakob H. Macke
2. Alessio Ansuini
3. Alessandro Laio
4. Davide Zoccolan

In Computer Vision and Pattern Recognition

1. Jakob H. Macke
2. Alessio Ansuini
3. Alessandro Laio
4. Davide Zoccolan

In Representation & Self-Supervised Learning

1. Jakob H. Macke
2. Alessio Ansuini
3. Alessandro Laio
4. Davide Zoccolan

Citations in the corpus

• Intrinsic dimension of data representations in deep neural networks (2019, methodology): “This phenomenon has been already observed by [27] and [6] on simplified datasets and architectures, and by [8] in the final, fully connected layers of AlexNet.”