Video-P2P: Video Editing with Cross-Attention Control
By Shaoteng Liu, Yuechen Zhang, Wenbo Li, Zhe Lin, Jiaya Jia. Computer Vision and Pattern Recognition, 2023
364 citations recorded by the source corpus.
Video-P2P is the first framework for real-world video editing with cross-attention control. While attention control has proven effective for image editing with pre-trained image generation models, there are currently no large-scale video generation models publicly available. Video-P2P addresses this limitation by adapting an image generation diffusion model to complete various video editing tasks. Specifically, we propose to first tune a Text-to-Set (T2S) model to complete an approximate inversion and then optimize a shared unconditional embedding to achieve accurate video inversion with a small memory cost. We further prove that it is crucial for consistent video editing. For attention control, we introduce a novel decoupled-guidance strategy, which uses different guidance strategies for the source and target prompts. The optimized unconditional embedding for the source prompt improves reconstruction ability, while an initialized unconditional embedding for the target prompt enhances editability. Incorporating the attention maps of these two branches enables detailed editing. These technical designs enable various text-driven editing applications, including word swap, prompt refinement, and attention re-weighting. Video-P2P works well on real-world videos for generating new characters while optimally preserving their original poses and scenes. It significantly outperforms previous approaches.
Subfields: Computer Vision and Pattern Recognition, Economics and Econometrics
View Paper: https://www.semanticscholar.org/paper/6283502d6900a0b403e2454b1cb1cf16ddefd5a7
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computer Vision and Pattern Recognition
- Chen Change Loy
- Luc Van Gool
- Ming-Hsuan Yang
- Dacheng Tao
- Ziwei Liu
- Xiaogang Wang
- Andrea Vedaldi
- Jitendra Malik
In Economics and Econometrics
Citations in the corpus
- VideoComposer: Compositional Video Synthesis with Motion Controllability (2023, methodology): “With the goal of editing videos guided by texts, VideoP2P [30] and vid2vid-zero [60] manipulate the cross-attention map, while Dreamix [33] proposes an image-video mixed fine-tuning strategy.”
- Pix2Video: Video Editing using Image Diffusion (2023, positive): “We note that our method is orthogonal and can adapt the Prompt-to-Prompt framework as the underlying text-to-image model as shown by the concurrent work [27].”
- VACE: All-in-One Video Creation and Editing (2025, positive): “Finally, we restructure the DiT model for VCU inputs, making it a versatile model for a wide range of video tasks.”
- TokenFlow: Consistent Diffusion Features for Consistent Video Editing (2023, background): “Concurrent works [34, 5, 20, 45, 26] leverage this property to achieve globally-coherent video edits.”
- Rerender A Video: Zero-Shot Text-Guided Video-to-Video Translation (2023, methodology): “Based on it, Edit-A-Video [35], VideoP2P [22] and vid2vid-zero [39] utilize Null-Text Inversion [24] for precise inversion to preserve the unedited region.”