AMP: Adaptive Masked Proxies for Few-Shot Segmentation

By Mennatullah Siam, Boris N. Oreshkin, Martin Jägersand. 2019

213 citations recorded by the source corpus.

Deep learning has thrived by training on large-scale datasets. However, in robotics applications sample efficiency is critical. We propose a novel adaptive masked proxies method that constructs the final segmentation layer weights from few labelled samples. It utilizes multi-resolution average pooling on base embeddings masked with the label to act as a positive proxy for the new class, while fusing it with the previously learned class signatures. Our method is evaluated on PASCAL-$5^i$ dataset and outperforms the state-of-the-art in the few-shot semantic segmentation. Unlike previous methods, our approach does not require a second branch to estimate parameters or prototypes, which enables it to be used with 2-stream motion and appearance based segmentation networks. We further propose a novel setup for evaluating continual learning of object segmentation which we name incremental PASCAL (iPASCAL) where our method outperforms the baseline method. Our code is publicly available at https://github.com/MSiam/AdaptiveMaskedProxies.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Meta & Continual Learning

Citations in the corpus

Language-driven Semantic Segmentation (2022, methodology): “Few-shot approaches (Shaban et al., 2017; Rakelly et al., 2018; Siam et al., 2019; Wang et al., 2019; Zhang et al., 2019; Nguyen & Todorovic, 2019; Liu et al., 2020b; Wang et al., 2020; Tian et al., 2020; Boudiaf et al., 2021; Min et al., 2021) offer ways to learn to segment novel classes based on…”
Incremental Few-Shot Instance Segmentation (2021, background): “In contrast, iMTFA uses weight-imprinting [25, 31] to keep an internal memory for the class representatives, and thus does not need this memory consumption at test or train time.”
Few-Shot Segmentation Without Meta-Learning: A Good Transductive Inference Is All You Need? (2020, background): “More recently, the dual-branch setting has been unified into a single-branch, employing the same embedding function for both the support and query sets [42, 30, 37, 38, 21].”
Progressive One-shot Human Parsing (2020, unknown): “AMP : we use masked proxies with multi-resolution weight imprinting technology and carefully tune a suitable learning rate as described in (Siam, Oreshkin, and Jagersand 2019).”
Hypercorrelation Squeeze for Few-Shot Segmenation (2021, methodology): “However, given only a few annotated training examples, the problem of poor generalization ability of the deep networks is yet the primary concern that many few-shot segmentation methods [10, 12, 13, 19, 33, 36, 37, 46, 54, 61, 63, 69, 70, 74, 75, 80, 83, 86, 87, 89] struggle to address.”
A Few Guidelines for Incremental Few-Shot Segmentation (2020, methodology): “As in previous few-shot segmentation works [38, 9, 41], we use the simplest strategy to extract class prototypes from multiple pixels, i.”
Learning What Not to Segment: A New Perspective on Few-Shot Segmentation (2022, methodology): “Fueled by the success of few-shot classiﬁcation, most existing FSS approaches strive to achieve generalization through meta-learning frameworks [23, 30–34, 36–38, 47, 55, 56, 58, 61, 62, 64–67].”
Few-Shot Semantic Segmentation with Democratic Attention Networks (2020, methodology): “Instead of using a prototype vector that loses the essential structure information [41, 40, 29], we establish pixel-to-pixel dense connections between the query image and the support image based on the graph attention [34].”