Pixel Difference Networks for Efficient Edge Detection

By Zhuo Su, Zitong Yu, Dewen Hu, Qing Liao, Qi Tian, Matti Pietikäinen, Li Liu. 2021

527 citations recorded by the source corpus.

Recently, deep Convolutional Neural Networks (CNNs) can achieve human-level performance in edge detection with the rich and abstract edge representation capacities. However, the high performance of CNN based edge detection is achieved with a large pretrained CNN backbone, which is memory and energy consuming. In addition, it is surprising that the previous wisdom from the traditional edge detectors, such as Canny, Sobel, and LBP are rarely investigated in the rapid-developing deep learning era. To address these issues, we propose a simple, lightweight yet effective architecture named Pixel Difference Network (PiDiNet) for efficient edge detection. PiDiNet adopts novel pixel difference convolutions that integrate the traditional edge detection operators into the popular convolutional operations in modern CNNs for enhanced performance on the task, which enjoys the best of both worlds. Extensive experiments on BSDS500, NYUD, and Multicue are provided to demonstrate its effectiveness, and its high training and inference efficiency. Surprisingly, when training from scratch with only the BSDS500 and VOC datasets, PiDiNet can surpass the recorded result of human perception (0.807 vs. 0.803 in ODS F-measure) on the BSDS500 dataset with 100 FPS and less than 1M parameters. A faster version of PiDiNet with less than 0.1M parameters can still achieve comparable performance among state of the arts with 200 FPS. Results on the NYUD and Multicue datasets show similar observations. The codes are available at https://github.com/zhuoinoulu/pidinet.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Ying Shan
2. Xintao Wang
3. Yujun Shen
4. Chen Change Loy
5. Haibin Ling
6. Luc Van Gool
7. Deli Zhao
8. Lianghua Huang

Citations in the corpus

• EDTER: Edge Detection with Transformer (2022, methodology): “For each type of annotations, we compare against the state-of-theart methods including HED [65], RCF [36], BDCN [22], DSCD [11], and PiDiNet [53].”
• VideoComposer: Compositional Video Synthesis with Motion Controllability (2023, methodology): “We extract sketch of the first video frame using PiDiNet [51] as the second spatial condition and encourage VideoComposer to synthesize temporal-consistent video according to the structure and texture within the single sketch. iii) Style.”
• T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Models (2023, methodology): “The adapter is trained with the edge prediction model PiDiNet [25].”
• Composer: Creative and Controllable Image Synthesis with Composable Conditions (2023, methodology): “Sketch: We apply an edge detection model (Su et al., 2021) followed by a sketch simplification algorithm (Simo-Serra et al.”
• DEA-Net: Single Image Dehazing Based on Detail-Enhanced Convolution and Content-Guided Attention (2023, methodology): “Given a hazy input image I ∈ R 3 × H × W , the goal of DEA-Net is to restore the corresponding haze-free image J ∈ R 3 × H × W .”