FreeDoM: Training-Free Energy-Guided Conditional Diffusion Model
By Jiwen Yu, Yinhuai Wang, Chen Zhao, Bernard Ghanem, Jian Zhang. IEEE International Conference on Computer Vision, 2023
344 citations recorded by the source corpus.
Recently, conditional diffusion models have gained popularity in numerous applications due to their exceptional generation ability. However, many existing methods are training-required. They need to train a time-dependent classifier or a condition-dependent score estimator, which increases the cost of constructing conditional diffusion models and is inconvenient to transfer across different conditions. Some current works aim to overcome this limitation by proposing training-free solutions, but most can only be applied to a specific category of tasks and not to more general conditions. In this work, we propose a training-Free conditional Diffusion Model (FreeDoM) used for various conditions. Specifically, we leverage off-the-shelf pretrained networks, such as a face detection model, to construct time-independent energy functions, which guide the generation process without requiring training. Furthermore, because the construction of the energy function is very flexible and adaptable to various conditions, our proposed FreeDoM has a broader range of applications than existing training-free methods. FreeDoM is advantageous in its simplicity, effectiveness, and low cost. Experiments demonstrate that FreeDoM is effective for various conditions and suitable for diffusion models of diverse data domains, including image and latent code domains. Code is available at https://github.com/vvictoryuki/FreeDoM.
Subfields: Computer Vision and Pattern Recognition, Radiology, Nuclear Medicine and Imaging, Signal Processing
View Paper: https://www.semanticscholar.org/paper/ce06533ecc98ba221a4db427738884c6a6af6eee
Citations in the corpus
- DragDiffusion: Harnessing Diffusion Models for Interactive Point-Based Image Editing (2023, unknown): “In contrast to previous methods [3, 10, 34, 53], which apply gradient descent on latents of multiple diffusion steps, our approach focuses on optimizing the latent of one appropriately selected step to conveniently achieve the desired editing results.”
- CLeaR: A Unified Framework for Resolving the Leakage-Degradation Dilemma in Style Transfer (2026, unknown): “For Energy-Guided Calibration, guidance is applied only in the refinement stage [48], i.e., t : T/ 10 → 0 , for 5 iterations with correction strength ρ = 0 .”