End-to-End Object Detection with Transformers

By Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, Sergey Zagoruyko. European Conference on Computer Vision, 2020

19,102 citations recorded by the source corpus.

We present a new method that views object detection as a direct set prediction problem. Our approach streamlines the detection pipeline, effectively removing the need for many hand-designed components like a non-maximum suppression procedure or anchor generation that explicitly encode our prior knowledge about the task. The main ingredients of the new framework, called DEtection TRansformer or DETR, are a set-based global loss that forces unique predictions via bipartite matching, and a transformer encoder-decoder architecture. Given a fixed small set of learned object queries, DETR reasons about the relations of the objects and the global image context to directly output the final set of predictions in parallel. The new model is conceptually simple and does not require a specialized library, unlike many other modern detectors. DETR demonstrates accuracy and run-time performance on par with the well-established and highly-optimized Faster RCNN baseline on the challenging COCO object detection dataset. Moreover, DETR can be easily generalized to produce panoptic segmentation in a unified manner. We show that it significantly outperforms competitive baselines. Training code and pretrained models are available at this https URL.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Model Architectures

View Paper: https://arxiv.org/abs/2005.12872

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Jianfeng Gao
  2. Mohit Bansal
  3. Quoc V. Le
  4. Dacheng Tao
  5. Yoshua Bengio
  6. Trevor Darrell
  7. Percy Liang
  8. Sergey Levine

In Computer Vision and Pattern Recognition

  1. Luc Van Gool
  2. Dacheng Tao
  3. Ming-Hsuan Yang
  4. Xiaogang Wang
  5. Trevor Darrell
  6. Jitendra Malik
  7. Andrew Zisserman
  8. Yi Yang

In Model Architectures

  1. Quoc V. Le
  2. Alan L. Yuille
  3. Jianfeng Gao
  4. Jiashi Feng
  5. Kaiming He
  6. Shuicheng Yan
  7. Chunhua Shen
  8. Dacheng Tao

Citations in the corpus