MonST3R: A Simple Approach for Estimating Geometry in the Presence of Motion
By Junyi Zhang, Charles Herrmann, Junhwa Hur, Varun Jampani, Trevor Darrell, Forrester Cole, Deqing Sun, Ming-Hsuan Yang. International Conference on Learning Representations, 2024
425 citations recorded by the source corpus.
Estimating geometry from dynamic scenes, where objects move and deform over time, remains a core challenge in computer vision. Current approaches often rely on multi-stage pipelines or global optimizations that decompose the problem into subtasks, like depth and flow, leading to complex systems prone to errors. In this paper, we present Motion DUSt3R (MonST3R), a novel geometry-first approach that directly estimates per-timestep geometry from dynamic scenes. Our key insight is that by simply estimating a pointmap for each timestep, we can effectively adapt DUST3R's representation, previously only used for static scenes, to dynamic scenes. However, this approach presents a significant challenge: the scarcity of suitable training data, namely dynamic, posed videos with depth labels. Despite this, we show that by posing the problem as a fine-tuning task, identifying several suitable datasets, and strategically training the model on this limited data, we can surprisingly enable the model to handle dynamics, even without an explicit motion representation. Based on this, we introduce new optimizations for several downstream video-specific tasks and demonstrate strong performance on video depth and camera pose estimation, outperforming prior work in terms of robustness and efficiency. Moreover, MonST3R shows promising results for primarily feed-forward 4D reconstruction.
Subfields: Computational Mechanics, Control and Systems Engineering, Geometry and Topology
View Paper: https://www.semanticscholar.org/paper/72ee22f4973522f70c46343a4e8b1cbcb346fa52
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computational Mechanics
- Angjoo Kanazawa
- Alexei A. Efros
- Bingyi Kang
- Aleksander Holynski
- Jiashi Feng
- Michael J. Black
- Jitendra Malik
- Andreas Geiger
In Control and Systems Engineering
Citations in the corpus
- Fast3R: Towards 3D Reconstruction of 1000+ Images in One Forward Pass (2025, neutral): “MASt3R [23] adds a local feature head to each decoder’s output, while MonST3R [69] does a data-driven exploration of dynamic scenes, but both are still fundamentally pairwise meth-21925”
- Continuous 3D Perception Model with Persistent State (2025, positive): “Following MonST3R [98], we evaluate camera pose estimation accuracy on Sintel [10], TUM dynamics [72], and Scan-Net [15] datasets.”
- Depth Anything 3: Recovering the Visual Space from Any Views (2025, positive): “Follow-up models extended this paradigm with multi-view inputs [10, 85, 94, 110], video input [19, 59, 94, 121], robust correspondence modeling [48], camera parameter injection [39, 43], large-scale SfM [18], SLAM applications [54], and view synthesis with 3D Gaussians [11, 13, 41, 79, 108, 122].”