Learning Distilled Collaboration Graph for Multi-Agent Perception

By Yiming Li, Shunli Ren, Pengxiang Wu, Siheng Chen, Chen Feng, Wenjun Zhang. Neural Information Processing Systems, 2021

430 citations recorded by the source corpus.

To promote better performance-bandwidth trade-off for multi-agent perception, we propose a novel distilled collaboration graph (DiscoGraph) to model trainable, pose-aware, and adaptive collaboration among agents. Our key novelties lie in two aspects. First, we propose a teacher-student framework to train DiscoGraph via knowledge distillation. The teacher model employs an early collaboration with holistic-view inputs; the student model is based on intermediate collaboration with single-view inputs. Our framework trains DiscoGraph by constraining post-collaboration feature maps in the student model to match the correspondences in the teacher model. Second, we propose a matrix-valued edge weight in DiscoGraph. In such a matrix, each element reflects the inter-agent attention at a specific spatial region, allowing an agent to adaptively highlight the informative regions. During inference, we only need to use the student model named as the distilled collaboration network (DiscoNet). Attributed to the teacher-student framework, multiple agents with the shared DiscoNet could collaboratively approach the performance of a hypothetical teacher model with a holistic view. Our approach is validated on V2X-Sim 1.0, a large-scale multi-agent perception dataset that we synthesized using CARLA and SUMO co-simulation. Our quantitative and qualitative experiments in multi-agent 3D object detection show that DiscoNet could not only achieve a better performance-bandwidth trade-off than the state-of-the-art collaborative perception methods, but also bring more straightforward design rationale. Our code is available on https://github.com/ai4ce/DiscoNet.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/6525519ed2bb3827c12878a6c3978726dcc90257

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

CoBEVT: Cooperative Bird's Eye View Semantic Segmentation with Sparse Transformers (2022, methodology): “We carefully examined the detection visualization comparisons between our method and the previous SOTA method DiscoNet.”
Where2comm: Communication-Efficient Collaborative Perception via Spatial Confidence Maps (2022, background): “The reason is that When2com employs a scalar to denote the agent-to-agent attention, which cannot distinguish which spatial area is more informative; DiscoNet employs a MLP-based fusion weight learning, which cannot well capture the complex collaboration attention; while Where2comm can zoom in to cr”
V2X-ViT: Vehicle-to-Everything Cooperative Perception with Vision Transformer (2022, methodology): “(a) OPV2V [44] (b) V2VNet [39] (c) DiscoNet [21] (d) V2X-ViT (ours)”
DAIR-V2X: A Large-Scale Dataset for Vehicle-Infrastructure Cooperative 3D Object Detection (2022, background): “…2020 real single vehicle 100M 0 0 10 ONCE [17] 2021 real single vehicle 7M 1M 417k 5 SYNTHIA [19] 2016 simulated single vehicle 213k 0 - 13 V2X-Sim [16] 2021 simulated multi-vehicle 0 10k 26.6k 2 highD [13] 2018 real infrastructure (UAV) In this paper, the Vehicle-Infrastructure Cooperative 3D…”