Unsupervised Learning of Edges

By Yin Li, Manohar Paluri, James M. Rehg, Piotr Doll'ar. 2015

99 citations recorded by the source corpus.

Data-driven approaches for edge detection have proven effective and achieve top results on modern benchmarks. However, all current data-driven edge detectors require manual supervision for training in the form of hand-labeled region segments or object boundaries. Specifically, human annotators mark semantically meaningful edges which are subsequently used for training. Is this form of strong, highlevel supervision actually necessary to learn to accurately detect edges? In this work we present a simple yet effective approach for training edge detectors without human supervision. To this end we utilize motion, and more specifically, the only input to our method is noisy semi-dense matches between frames. We begin with only a rudimentary knowledge of edges (in the form of image gradients), and alternate between improving motion estimation and edge detection in turn. Using a large corpus of video data, we show that edge detectors trained using our unsupervised scheme approach the performance of the same methods trained with full supervision (within 3-5%). Finally, we show that when using a deep network for the edge detector, our approach provides a novel pre-training scheme for object detection.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Data-Efficient Image Recognition with Contrastive Predictive Coding (2019, methodology): “For some experiments, we also ﬁne-tune the encoder network (blue) for the classiﬁcation task. have leveraged object tracking [61], frame ordering [43], and object boundary cues [40, 50].”
Learning Features by Watching Objects Move (2016, methodology): “[26] use motion boundary detection to bootstrap a ConvNet-based contour detector, but find that this does not lead to good feature representations.”
EdgeConnect: Generative Image Inpainting with Adversarial Edge Learning (2019, background): “There is a large body of literature that addresses “Image-to-Edges” problems [5, 10, 26, 29].”
Multi-task Self-Supervised Visual Learning (2017, background): “In the last few years, a considerable number of such tasks have been proposed [1, 2, 6, 7, 8, 17, 20, 21, 23, 25, 26, 27, 28, 29, 31, 39, 40, 42, 43, 46, 47], such as asking a neural network to colorize grayscale images, ﬁll in image holes, solve jigsaw puzzles made from image patches, or predict…”
Generating Videos with Scene Dynamics (2016, background): “Finally, this paper is related to a growing body of work that capitalizes on large amounts of unlabeled video for visual recognition tasks [13, 41, 37, 32, 45, 5, 11, 36, 46, 47, 33, 14, 39, 44].”
Holistically-Nested Edge Detection (2015, background): “The task of edge and object boundary detection is intrinsically challenging.”
Generative Image Modeling Using Style and Structure Adversarial Networks (2016, background): “There are two common approaches for unsupervised learning: (a) using a discriminative framework with auxiliary tasks where supervision comes for free, such as context prediction [1,2] or temporal embedding [3,4,5,6,7,8]; (b) using a generative framework where the underlying model is compositional an”
Transitive Invariance for Self-Supervised Visual Representation Learning (2017, background): “Visual invariance can be captured by the same instance/scene taken in a sequence of video frames [56, 48, 25, 1, 37, 52, 32, 39].”
Self-Supervised Spatiotemporal Feature Learning via Video Rotation Prediction. (2018, methodology): “…two categories: one is the texture based methods, which utilize texture information of images as supervision, such as the boundary of the objects [28, 38], the context of images [25, 36], and the similarity of two patches from an image [6, 15, 33, 34, 48], and the other is the temporal based…”
Self-supervised Spatiotemporal Feature Learning by Video Geometric Transformations (2018, methodology): “Based on the pretext tasks, these methods fall into two categories: texture-based methods which mainly utilize the texture information of images as the supervision information such as the boundary of the objects [22, 34], the context of images [18, 31], and the similarity of two patches from an imag”
Space-Time Correspondence as a Contrastive Random Walk (2020, background): “For video, temporal information makes for natural pretext tasks, including future prediction [26, 74, 52, 47, 50], arrow of time [55, 89], motion estimation [1, 36, 78, 45] or audio [62, 2, 61, 41].”
Efficient Visual Pretraining with Contrastive Detection (2021, background): “While perceptual grouping has long been used for representation learning, often relying on coherent motion in videos [37, 45, 61], it has only recently been combined with contrastive learning [30, 59, 68].”
Richer Convolutional Features for Edge Detection (2016, background): “[35] proposed a complex model for unsupervised learning of edge detection, but the performance is worse than training on the limited BSDS500 dataset.”
Self-Supervised Visual Feature Learning With Deep Neural Networks: A Survey (2019, methodology): “proposed to learn features by training a ConvNet for edge prediction while labels are motion edges obtained from flow fields from videos [47].”
EdgeConnect: Structure Guided Image Inpainting using Edge Prediction (2019, background): “There is a large body of literature that addresses “Image-to-Edges” problems [6, 11, 29, 32].”
Self-Supervised Visual Representation Learning from Hierarchical Grouping (2020, unknown): “Serving as a noteworthy example is Li et al. ’s [30] approach of using motion as a readily available, automatically-derived, supervisory signal for learning to detect contours in static images.”
A Survey on Self-Supervised Learning: Algorithms, Applications, and Future Trends (2023, background): “The advancement of SSL [3], [4], [15], [16], [17], [18], [19], [20], [21], [22], [23], [24] has exhibitedrapidprogress, capturing signiﬁcant attention within the research community (Fig.”
Learning Correspondence From the Cycle-Consistency of Time (2019, unknown): “Learning representations from video using time as supervision has been extensively studied, both as future prediction task [15, 60, 44, 42] as well as motion estimation [2, 25, 63, 38, 40].”
Unsupervised Deep Learning for Optical Flow Estimation (2017, methodology): “Is pixel-level supervision indispensable for learning optical flow? Recent work on learning from video has shown that via some quality control, effective feature representation (Wang and Gupta 2015; Li et al. 2016) and even cross-instance key-point matching (Zhou et al.”