PointOdyssey: A Large-Scale Synthetic Dataset for Long-Term Point Tracking

By Yang Zheng, Adam W. Harley, Bokui (William) Shen, Gordon Wetzstein, Leonidas J. Guibas. IEEE International Conference on Computer Vision, 2023

294 citations recorded by the source corpus.

We introduce PointOdyssey, a large-scale synthetic dataset, and data generation framework, for the training and evaluation of long-term fine-grained tracking algorithms. Our goal is to advance the state-of-the-art by placing emphasis on long videos with naturalistic motion. Toward the goal of naturalism, we animate deformable characters using real-world motion capture data, we build 3D scenes to match the motion capture environments, and we render camera viewpoints using trajectories mined via structure-from-motion on real videos. We create combinatorial diversity by randomizing character appearance, motion profiles, materials, lighting, 3D assets, and atmospheric effects. Our dataset currently includes 104 videos, averaging 2,000 frames long, with orders of magnitude more correspondence annotations than prior work. We show that existing methods can be trained from scratch in our dataset and outperform the published variants. Finally, we introduce modifications to the PIPs point tracking method, greatly widening its temporal receptive field, which improves its performance on PointOdyssey as well as on two real-world benchmarks. Our data and code are publicly available at: https://pointodyssey.com

Subfields: Computational Mechanics, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/f84a328afa81ecbd651636633bea0618045d0dfc

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computer Vision and Pattern Recognition

Citations in the corpus

Depth Anything 3: Recovering the Visual Space from Any Views (2025, unknown): “…[98], IRS [93], vKITTI2 [9], BlendedMVS [115], SPRING [56], MVSSynth [38], UnrealStereo4K [123], GTA-SfM [92], TauAgent [27], KenBurns [60], MatrixCity [50], EDEN [47], ReplicaGSO [82], UrbanSyn [32], PointOdyssey [127], Structured3D [125], Objaverse [17], Trel-lis [104], and OmniObject [102].”
CoTracker3: Simpler and Better Point Tracking by Pseudo-Labeling Real Videos (2025, unknown): “Training data: (Kub) Kubric [11], (PO) Point Odyssey [44].”
CoTracker: It is Better to Track Together (2023, methodology): “CoTracker is trained with a batch size of 32, distributed across 32 GPUs.”
Fast3R: Towards 3D Reconstruction of 1000+ Images in One Forward Pass (2025, unknown): “Fast3R can be trained to achieve such results by finetuning a 16 static views checkpoint on the PointOdyssey [73] and TartanAir [62] datasets, consisting of 110 dynamic and 150 static scenes, respectively.”
VGGT: Visual Geometry Grounded Transformer (2025, unknown): “Hence, while the feed-forward mode of VGGT outperforms all previous alternatives (whether they are feed-forward or not), there is still room for improvement since post-optimization still brings beneﬁts.”
MonST3R: A Simple Approach for Estimating Geometry in the Presence of Motion (2024, methodology): “During training, we sample the datasets asymmetrically to place extra weight on PointOdyssey (more dynamic, articulated objects) and less weight on TartanAir (good scene diversity but static) and Waymo (a highly specialized domain).”