Generative Flows on Discrete State-Spaces: Enabling Multimodal Flows with Applications to Protein Co-Design

By Andrew Campbell, Jason Yim, R. Barzilay, Tom Rainforth, Tommi S. Jaakkola. International Conference on Machine Learning, 2024

330 citations recorded by the source corpus.

Combining discrete and continuous data is an important capability for generative models. We present Discrete Flow Models (DFMs), a new flow-based model of discrete data that provides the missing link in enabling flow-based generative models to be applied to multimodal continuous and discrete data problems. Our key insight is that the discrete equivalent of continuous space flow matching can be realized using Continuous Time Markov Chains. DFMs benefit from a simple derivation that includes discrete diffusion models as a specific instance while allowing improved performance over existing diffusion-based approaches. We utilize our DFMs method to build a multimodal flow-based modeling framework. We apply this capability to the task of protein co-design, wherein we learn a model for jointly generating protein structure and sequence. Our approach achieves state-of-the-art co-design performance while allowing the same multimodal model to be used for flexible generation of the sequence or structure.

View Paper: https://www.semanticscholar.org/paper/b30ce57128b672945b3e24f98aee63b2b3881ee0

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

1. Tal Remez
2. Yossi Adi
3. Itai Gat
4. Gabriel Synnaeve
5. Felix Kreuk
6. Neta Shaul
7. Yaron Lipman
8. Ricky T. Q. Chen

Citations in the corpus

• Simplified and Generalized Masked Diffusion for Discrete Data (2024, unknown): “All other configurations are kept the same.”
• Discrete Flow Matching (2024, methodology): “Linear Scheduler (Campbell et al., 2024).”
• Simple and Effective Masked Diffusion Language Models (2024, unknown): “…z s and z t , where s t , let A.1.1 q ( z t | z s ) Thus, the marginals in (3) correspond to the following forward process: The above equation indicates that during each diffusion step from s → t , a fraction (1 − α t | s ) of the probability mass is transferred to the prior distribution π .”