Unsupervised part representation by Flow Capsules
By Sara Sabour, Andrea Tagliasacchi, Soroosh Yazdani, Geoffrey E. Hinton, David J. Fleet. 2020
41 citations recorded by the source corpus.
Capsule networks are designed to parse an image into a hierarchy of objects, parts and relations. While promising, they remain limited by an inability to learn effective low level part descriptions. To address this issue we propose a novel self-supervised method for learning part descriptors of an image. During training, we exploit motion as a powerful perceptual cue for part definition, using an expressive decoder for part generation and layered image formation with occlusion. Experiments demonstrate robust part discovery in the presence of multiple objects, cluttered backgrounds, and significant occlusion. The resulting part descriptors, a.k.a. part capsules, are decoded into shape masks, filling in occluded pixels, along with relative depth on single images. We also report unsupervised object classification using our capsule parts in a stacked capsule autoencoder.
Subfields: Computer Vision and Pattern Recognition
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Andrea Vedaldi
- Dacheng Tao
- Geoffrey E. Hinton
- Ming-Hsuan Yang
- Jitendra Malik
- Xiaogang Wang
- Trevor Darrell
Citations in the corpus
- How to Represent Part-Whole Hierarchies in a Neural Network (2021, background): “…of a static image, but the architecture is motivated by the need to deal with video and learning from video is often much easier than learning from static images [Sabour et al., 2021], so I will briefly discuss the simplest temporal extension which is to a single fixation of a time-varying image.”
- Conditional Object-Centric Learning from Video (2021, methodology): “A recent related method, FlowCaps (Sabour et al., 2020), similarly proposed to use optical flow in a multi-object model.”
- Canonical Capsules: Unsupervised Capsules in Canonical Pose (2020, background): “Results are better appreciated in our supplementary material , where we visualize the performance as we continuously traverse SE (3) .”