CoTracker: It is Better to Track Together
By Nikita Karaev, Ignacio Rocco, Benjamin Graham, Natalia Neverova, Andrea Vedaldi, Christian Rupprecht. European Conference on Computer Vision, 2023
618 citations recorded by the source corpus.
We introduce CoTracker, a transformer-based model that tracks a large number of 2D points in long video sequences. Differently from most existing approaches that track points independently, CoTracker tracks them jointly, accounting for their dependencies. We show that joint tracking significantly improves tracking accuracy and robustness, and allows CoTracker to track occluded points and points outside of the camera view. We also introduce several innovations for this class of trackers, including using token proxies that significantly improve memory efficiency and allow CoTracker to track 70k points jointly and simultaneously at inference on a single GPU. CoTracker is an online algorithm that operates causally on short windows. However, it is trained utilizing unrolled windows as a recurrent network, maintaining tracks for long periods of time even when points are occluded or leave the field of view. Quantitatively, CoTracker substantially outperforms prior trackers on standard point-tracking benchmarks.
Subfields: Computer Vision and Pattern Recognition
View Paper: https://www.semanticscholar.org/paper/fa85633fb74d071911fe64d952afe47adc9928d6
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Andrea Vedaldi
- Andrew Zisserman
- Jitendra Malik
- Xiaogang Wang
- Dacheng Tao
- Bernt Schiele
- Trevor Darrell
Citations in the corpus
- CoTracker3: Simpler and Better Point Tracking by Pseudo-Labeling Real Videos (2025, positive): “Co-Tracker [16] proposed a transformer architecture that tracks multiple points jointly, achieving further gains in tracking quality, particularly for points that are partially occluded in the video.”
- VGGT: Visual Geometry Grounded Transformer (2025, neutral): “CoTracker [37, 38] utilized correlations between different points to track through occlusions, while DOT [41] enabled dense tracking through occlusions.”
- Dynamic 3D Gaussians: Tracking by Persistent Dynamic View Synthesis (2023, background): “Most of these prior approaches [6, 7, 12, 16, 47] are deep learning based and work by being trained on a large-dataset of ground-truth point tracks (often using synthetic training data).”