RAFT: Recurrent All-Pairs Field Transforms for Optical Flow

By Zachary Teed, Jia Deng. 2020

3,984 citations recorded by the source corpus.

We introduce Recurrent All-Pairs Field Transforms (RAFT), a new deep network architecture for optical flow. RAFT extracts per-pixel features, builds multi-scale 4D correlation volumes for all pairs of pixels, and iteratively updates a flow field through a recurrent unit that performs lookups on the correlation volumes. RAFT achieves state-of-the-art performance on the KITTI and Sintel datasets. In addition, RAFT has strong cross-dataset generalization as well as high efficiency in inference time, training speed, and parameter count.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Perceiver IO: A General Architecture for Structured Inputs & Outputs (2021, methodology): “We then upsample this flow for a given pixel in the high-resolution flow map by applying attention over a neighboring 3x3 block of the low-resolution flow map, following the uppsampling approach in RAFT [84].”
Deformable DETR: Deformable Transformers for End-to-End Object Detection (2020, methodology): “This is inspired by the iterative reﬁnement developed in optical ﬂow estimation (Teed & Deng, 2020).”
Is Space-Time Attention All You Need for Video Understanding? (2021, background): “However, in the video domain, 2D or 3D convolutions still represent the core operators for spatiotemporal feature learning across different video tasks (Feichtenhofer et al., 2019a; Teed & Deng, 2020; Bertasius & Torresani, 2020).”
NeRF-SLAM: Real-Time Dense Monocular SLAM with Neural Radiance Fields (2022, methodology): “Starting from a sequence of images, Droid-SLAM first computes the dense optical-flow p ij between pairs of frames i and j , using a similar architecture to Raft [1].”
CoTracker3: Simpler and Better Point Tracking by Pseudo-Labeling Real Videos (2025, positive): “Inspired by RAFT [34], an optical flow algorithm, PIPs extracts correlation maps be-tween frames and feeds them into a network to refine the track estimates.”
Step1X-Edit: A Practical Framework for General Image Editing (2025, positive): “We use BiRefNet [72] and RAFT [54] for foreground-background separation and optical flow estimation.”
3D-VLA: A 3D Vision-Language-Action Generative World Model (2024, positive): “Additionally, to better utilize video data, we use RAFT (Teed & Deng, 2020) for optical flow estimation.”
VACE: All-in-One Video Creation and Editing (2025, positive): “In the actual training process, the construction for different tasks also needs to be tailored according to the characteristics of each task: 1) For some controllable video generation tasks, we pre-extract depth [51], scribble [6], pose [5, 72], and optical flow [65] from the filtered videos.”
Grounding Image Matching in 3D with MASt3R (2024, background): “Very reminiscent of optical flow approaches [22,40,42,79,80,86], they are usually employing coarse-to-fine schemes to decrease computational complexity.”
OpenVid-1M: A Large-Scale High-Quality Dataset for Text-to-video Generation (2024, methodology): “2) Motion evaluation : SVD utilizes the traditional Farneback optical flow method and RAFT [32] to estimate optical flows.”
AutoFlow: Learning a Better Training Set for Optical Flow (2021, methodology): “However, state-of-the-art, self-supervised methods still lag behind supervised ones, e.g., models pre-trained on FlyingChairs and FlyingThings [48, 51] are more accurate on Sintel than models [21, 28] trained on Sintel image pairs using self-supervised loss.”
Learning to Estimate Hidden Motions with Global Motion Aggregation (2021, methodology): “Reference Frame Matching Frame Flow (RAFT [38]) Flow (Ours) Flow (Ground Truth) Error (RAFT [38]) Error (Ours)”
Unsupervised Semantic Segmentation by Distilling Feature Correspondences (2022, methodology): “To use this information, we focus on the “correlation volume” (Teed & Deng, 2020) between the dense feature maps.”
Human Preference Score: Better Aligning Text-to-image Models with Human Preference (2023, background): “Datasets of generated images play a vital role in computer vision tasks that has difficulty in ground-truth acquisition, such as optical flow estimation [24, 7, 4, 44, 18, 39, 40].”
CoTracker: It is Better to Track Together (2023, background): “3 we demonstrate the importance of tracking points jointly, which is the main motivation of CoTracker.”
CameraCtrl: Enabling Camera Control for Text-to-Video Generation (2024, positive): “Then, following the dynamic degree in VBench, we use RAFT (Teed & Deng, 2020) to estimate the optical flow, and only keeping the estimated flows belongs to the main object.”
TokenFlow: Consistent Diffusion Features for Consistent Video Editing (2023, methodology): “Following [5, 22], temporal consistency is measured by computing the optical ﬂow of the original video using [49], warping the edited frames according to it, and measuring the warping error.”