AnyDoor: Zero-shot Object-level Image Customization

By Xi Chen, Lianghua Huang, Yu Liu, Yujun Shen, Deli Zhao, Hengshuang Zhao. Computer Vision and Pattern Recognition, 2023

503 citations recorded by the source corpus.

This work presents AnyDoor, a diffusion-based image generator with the power to teleport target objects to new scenes at user-specified locations with desired shapes. Instead of tuning parameters for each object, our model is trained only once and effortlessly generalizes to diverse object-scene combinations at the inference stage. Such a challenging zero-shot setting requires an adequate characterization of a certain object. To this end, we complement the commonly used identity feature with detail features, which are carefully designed to maintain appearance details yet allow versatile local variations (e.g., lighting, orientation, posture, etc.), supporting the object in favorably blending with different surroundings. We further propose to borrow knowledge from video datasets, where we can observe various forms (i.e., along the time axis) of a single object, leading to stronger model generalizability and robustness. Extensive experiments demonstrate the superiority of our approach over existing alternatives as well as its great potential in real-world applications, such as virtual try-on, shape editing, and object swapping. Code is released at github.com/ali-vilab/AnyDoor.

Subfields: Biophysics, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/9b4e61dda9db6317afae1bd4a12356d00769d9f3

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Biophysics

1. Deli Zhao
2. Yujun Shen
3. Daniel Cohen-Or
4. Li Luo

In Computer Vision and Pattern Recognition

1. Yujun Shen
2. Deli Zhao
3. Jingren Zhou
4. Hangjie Yuan
5. Yingya Zhang
6. Shiwei Zhang
7. Xiang Wang
8. Dayou Chen

Citations in the corpus

• VACE: All-in-One Video Creation and Editing (2025, unknown): “Mean-while Mean-while, to meet the diverse needs of visual media production and to enhance”
• VideoComposer: Compositional Video Synthesis with Motion Controllability (2023, background): “Recently, research efforts on image synthesis have shifted from utilizing GANs [17], VAEs [29], and flow models [13] to diffusion models [21, 50, 54, 73] due to more stable training, enhanced sample quality, and increased flexibility in a conditional generation.”
• InstantID: Zero-shot Identity-Preserving Generation in Seconds (2024, background): “A concurrent work, Anydoor [2], complements the commonly used identity feature with detail features that are designed to maintain texture details while allowing versatile local variations.”
• PhotoMaker: Customizing Realistic Human Photos via Stacked ID Embedding (2023, unknown): “Recently, some studies [10, 11, 29, 42, 43, 62, 70] attempt to perform personalized generation using a single image with a single forward pass, significantly accelerating the personalization process.”