Learning Deep Parsimonious Representations

By Renjie Liao, Alexander Gerhard Schwing, Richard S. Zemel, Raquel Urtasun. 2016

101 citations recorded by the source corpus.

Subfields: Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Representation & Self-Supervised Learning

Citations in the corpus

Unsupervised Representation Learning by Predicting Image Rotations (2018, methodology): “Other successful cases of unsupervised feature learning are clustering based methods (Dosovitskiy et al., 2014; Liao et al., 2016; Yang et al., 2016), reconstruction based methods (Bengio et al., 2007; Huang et al., 2007; Masci et al., 2011), and methods that involve learning generative…”
Unsupervised Embedding Learning via Invariant and Spreading Instance Feature (2019, methodology): “In comparison, general unsupervised feature learning usually aims at learning a good “intermediate” feature representation from unlabelled data [6, 26, 31, 34].”
Prototypical Networks for Few-shot Learning (2017, methodology): “We also compare to a recent clustering approach [19] which trains an SVM on a learned feature space obtained by fine-”
Deep Clustering for Unsupervised Learning of Visual Features (2018, background): “Other clustering losses [3, 16, 35, 66, 68] have been considered to jointly learn convnet features and image clusters but they have never been tested on a scale to allow a thorough study on modern convnet architectures.”
Unsupervised Learning by Predicting Noise (2017, background): “Some attempts were made with retrieval based approaches (Doso-vitskiy et al., 2014) and clustering (Yang et al., 2016; Liao et al., 2016), but they are hard to scale and have only been tested on small datasets.”
Prototypical Contrastive Learning of Unsupervised Representations (2020, background): “Approaches in [14, 15, 16, 17, 18, 19] jointly learn image embeddings and cluster assignments, but they have not shown the ability to learn transferable representations from a large scale of images.”
Unsupervised Learning of Neural Networks to Explain Neural Networks (2018, background): “A new trend related to network interpretability is to learn networks with disentangled, interpretable representations [12, 31, 21].”
Interpretable Convolutional Neural Networks (2017, background): “[16] learned compact CNN representations, but they did not make filters obtain explicit part-level or texture-level semantics.”
Pipe-SGD: A Decentralized Pipelined SGD Framework for Distributed Deep Net Training (2018, methodology): “The simple AlexNet-style CIFAR100 architecture in [32] is used for benchmarking this datasets.”
Smooth Neighbors on Teacher Graphs for Semi-Supervised Learning (2017, background): “the top layer maps the inputs into a low-dimensional feature space [5, 42, 29].”
Normalizing the Normalizers: Comparing and Extending Network Normalization Schemes (2016, methodology): “Liao et al. (2016a) proposed to accumulate the normalization statistics over the entire training phase, and showed that this can speed up training in recurrent and online learning without a deteriorating effect on the performance.”
Leveraging Large-Scale Uncurated Data for Unsupervised Pre-training of Visual Features (2019, background): “Various attempts have been made to train convnets using clustering [2, 3, 6, 29, 48, 52, 53].”
Learn from Others and Be Yourself in Heterogeneous Federated Learning (2022, methodology): “For the heterogeneity problem, we take inspiration from the self-supervised learning [5, 6, 11, 13, 18, 25, 49, 91, 94].”
Visual Concepts and Compositional Voting (2017, result): “This is similar to the regularization method reported in [23].”
Compositional Convolutional Neural Networks: A Robust and Interpretable Model for Object Recognition Under Occlusion (2020, methodology): “reas our graph is learned from class supervision only and still learns a meaningful human-interpretable representation. 2.3 Deep Compositional Models in Computer Vision An early work from Liao et al. [37] proposes to integrate compositionality into DCNNs by regularizing the feature representations o”
Convolutional Bipartite Attractor Networks (2019, background): “For example, a few recent articles have proposed incorporating attractor-like dynamics (clustering methods, denoising autoencoders) to clean up internal states and improve the robustness of deep models [21, 25].”
Unsupervised Image Classification for Deep Representation Learning (2020, methodology): “ustering-based methods Clustering-based methods are mostly related to our proposed method. Coates et al. [7] is the rst to pretrain CNNs via clustering in a layer-by-layer manner. The following works [39,38,25,2] are also motivated to jointly cluster images and learn visual features. Among them, Dee”
DEEP CLUSTERING AND CONVENTIONAL NETWORKS FOR MUSIC SEPARATION: STRONGER TOGETHER (2016, background): “In recent work it has been shown that forcing deep network activations to cluster well can improve the resulting test performance [12].”
Unsupervised Pre-Training of Image Features on Non-Curated Data (2019, unknown): “Various attempts have been made to train convnets using clustering [2, 3, 6, 29, 45, 49, 50].”