InstantID: Zero-shot Identity-Preserving Generation in Seconds

By Qixun Wang, Xu Bai, Haofan Wang, Zekui Qin, Anthony Chen. arXiv.org, 2024

501 citations recorded by the source corpus.

There has been significant progress in personalized image synthesis with methods such as Textual Inversion, DreamBooth, and LoRA. Yet, their real-world applicability is hindered by high storage demands, lengthy fine-tuning processes, and the need for multiple reference images. Conversely, existing ID embedding-based methods, while requiring only a single forward inference, face challenges: they either necessitate extensive fine-tuning across numerous model parameters, lack compatibility with community pre-trained models, or fail to maintain high face fidelity. Addressing these limitations, we introduce InstantID, a powerful diffusion model-based solution. Our plug-and-play module adeptly handles image personalization in various styles using just a single facial image, while ensuring high fidelity. To achieve this, we design a novel IdentityNet by imposing strong semantic and weak spatial conditions, integrating facial and landmark images with textual prompts to steer the image generation. InstantID demonstrates exceptional performance and efficiency, proving highly beneficial in real-world applications where identity preservation is paramount. Moreover, our work seamlessly integrates with popular pre-trained text-to-image diffusion models like SD1.5 and SDXL, serving as an adaptable plugin. Our codes and pre-trained checkpoints will be available at https://github.com/InstantID/InstantID.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/0f9b66c9208b11369e9d94d85b7dc23bcc5115e9

Citations in the corpus

• OmniGen: Unified Image Generation (2024, unknown): “Furthermore, OmniGen jointly models text and images within a single model, rather than independently modeling different input conditions with separate encoders as in existing works [8, 64, 68, 70, 72] which lacks interaction between different modality conditions.”
• OminiControl: Minimal and Universal Control for Diffusion Transformer (2024, unknown): “To address this limitation, some works have explored image-based control methods [2, 14, 15, 18, 26– 30, 37, 42, 46, 50, 52, 54, 56, 57, 57, 60, 61, 63], enabling users to specify their intentions through reference images or visual hints, offering more precise guidance than text alone [44].”