Emerging Properties in Unified Multimodal Pretraining

By Chaorui Deng, Deyao Zhu, Kunchang Li, Chenhui Gou, Feng Li, Zeyu Wang, Shu Zhong, Weihao Yu, Xiao-Ping Nie, Ziang Song and 2 more. arXiv.org, 2025

761 citations recorded by the source corpus.

Unifying multimodal understanding and generation has shown impressive capabilities in cutting-edge proprietary systems. In this work, we introduce BAGEL, an open-source foundational model that natively supports multimodal understanding and generation. BAGEL is a unified, decoder-only model pretrained on trillions of tokens curated from large-scale interleaved text, image, video, and web data. When scaled with such diverse multimodal interleaved data, BAGEL exhibits emerging capabilities in complex multimodal reasoning. As a result, it significantly outperforms open-source unified models in both multimodal generation and understanding across standard benchmarks, while exhibiting advanced multimodal reasoning abilities such as free-form image manipulation, future frame prediction, 3D manipulation, and world navigation. In the hope of facilitating further opportunities for multimodal research, we share the key findings, pretraining details, data creation protocal, and release our code and checkpoints to the community. The project page is at https://bagel-ai.org/

View Paper: https://www.semanticscholar.org/paper/406a577425c36e05163ee3bf448d65a6eb480ab3

Citations in the corpus

• Qwen-Image Technical Report (2025, unknown): “…et al., 2025) 5.96 5.89 5.06 - - - OmniGen2 (Wu et al., 2025b) 7.16 6.77 6.41 - - - Gemini 2.0 (DeepMind, 2025) 6.73 6.61 6.32 5.43 6.78 5.36 BAGEL (Deng et al., 2025) 7 Image ranks at the top of both the English and Chinese leaderboards, demonstrating strong editing capability and generalization…”
• FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Space (2025, unknown): “IntelligentBench [9] remains unavailable with only 300 examples of uncertain task coverage.”