Continuous 3D Perception Model with Persistent State

By Qianqian Wang, Yifei Zhang, Aleksander Holynski, Alexei A. Efros, Angjoo Kanazawa. Computer Vision and Pattern Recognition, 2025

511 citations recorded by the source corpus.

We present a unified framework capable of solving a broad range of 3D tasks. Our approach features a stateful recurrent model that continuously updates its state representation with each new observation. Given a stream of images, this evolving state can be used to generate metric-scale pointmaps (per-pixel 3D points) for each new input in an online fashion. These pointmaps reside within a common coordinate system, and can be accumulated into a coherent, dense scene reconstruction that updates as new images arrive. Our model, called CUT3R (Continuous Updating Transformer for 3D Reconstruction), captures rich priors of real-world scenes: not only can it predict accurate pointmaps from image observations, but it can also infer unseen regions of the scene by probing at virtual, unobserved views. Our method is simple yet highly flexible, naturally accepting varying length of images that may be either video streams or unordered photo collections, containing both static and dynamic content. We evaluate our method on various 3D/4D tasks and demonstrate competitive or state-of-the-art performance in each. Project page: https://cut3r.github.io/.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/4d969386c65de5c892f6477314a62cda523772ad

Citations in the corpus

• Depth Anything 3: Recovering the Visual Space from Any Views (2025, unknown): “Follow-up models extended this paradigm with multi-view inputs [10, 85, 94, 110], video input [19, 59, 94, 121], robust correspondence modeling [48], camera parameter injection [39, 43], large-scale SfM [18], SLAM applications [54], and view synthesis with 3D Gaussians [11, 13, 41, 79, 108, 122].”
• VGGT: Visual Geometry Grounded Transformer (2025, unknown): “Some concurrent works [73, 85, 96, 105] explore replacing DUSt3R’s test-time optimization with neural networks, though these attempts achieve only suboptimal or comparable performance to DUSt3R.”