TAPIR: Tracking Any Point with per-frame Initialization and temporal Refinement
By Carl Doersch, Yi Yang, Mel Vecerík, Dilara Gokay, Ankush Gupta, Yusuf Aytar, João Carreira, Andrew Zisserman. IEEE International Conference on Computer Vision, 2023
336 citations recorded by the source corpus.
We present a novel model for Tracking Any Point (TAP) that effectively tracks any queried point on any physical surface throughout a video sequence. Our approach employs two stages: (1) a matching stage, which independently locates a suitable candidate point match for the query point on every other frame, and (2) a refinement stage, which updates both the trajectory and query features based on local correlations. The resulting model surpasses all baseline methods by a significant margin on the TAP-Vid benchmark, as demonstrated by an approximate 20% absolute average Jaccard (AJ) improvement on DAVIS. Our model facilitates fast inference on long and high-resolution video sequences. On a modern GPU, our implementation has the capacity to track points faster than real-time. Given the high-quality trajectories extracted from a large dataset, we demonstrate a proof-of-concept diffusion model which generates trajectories from static images, enabling plausible animations. Visualizations, source code, and pretrained models can be found at https://deepmind-tapir.github.io.
Subfields: Computer Vision and Pattern Recognition
View Paper: https://www.semanticscholar.org/paper/fc98d0e981f193b159d8fab5343f094cf06dcf20
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Andrea Vedaldi
- Andrew Zisserman
- Xiaogang Wang
- Jitendra Malik
- Dacheng Tao
- Trevor Darrell
- Bernt Schiele
Citations in the corpus
- CoTracker: It is Better to Track Together (2023, background): “Our architecture works well for tracking single points and excels for groups of points, obtaining state-of-the-art tracking performance in several benchmarks.”
- Dynamic 3D Gaussians: Tracking by Persistent Dynamic View Synthesis (2023, background): “Most of these prior approaches [6, 7, 12, 16, 47] are deep learning based and work by being trained on a large-dataset of ground-truth point tracks (often using synthetic training data).”
- CoTracker3: Simpler and Better Point Tracking by Pseudo-Labeling Real Videos (2025, neutral): “Our set of teachers consists of our proposed models CoTracker3 online and CoTracker3 offline, CoTracker [16], and TAPIR [7].”
- VGGT: Visual Geometry Grounded Transformer (2025, neutral): “TAP-Vid [13] proposed three benchmarks for this task, and a simple baseline method later improved in TAPIR [14].”