PaliGemma: A versatile 3B VLM for transfer

By L. Beyer, Andreas Steiner, André Susano Pinto, Alexander Kolesnikov, Xiao Wang, Daniel M. Salz, Maxim Neumann, Ibrahim M. Alabdulmohsin, Michael Tschannen, Emanuele Bugliarello and 25 more. arXiv.org, 2024

728 citations recorded by the source corpus.

PaliGemma is an open Vision-Language Model (VLM) that is based on the SigLIP-So400m vision encoder and the Gemma-2B language model. It is trained to be a versatile and broadly knowledgeable base model that is effective to transfer. It achieves strong performance on a wide variety of open-world tasks. We evaluate PaliGemma on almost 40 diverse tasks including standard VLM benchmarks, but also more specialized tasks such as remote-sensing and segmentation.

Subfields: Electrical and Electronic Engineering

View Paper: https://www.semanticscholar.org/paper/440d7944dc701dbe4030d3fbad49036700fba788

Citations in the corpus

π0: A Vision-Language-Action Flow Model for General Robot Control (2024, neutral): “In contrast to these works, our model uses a pre-trained VLM backbone [5].”
LLaVA-OneVision: Easy Visual Task Transfer (2024, neutral): “For comprehensive investigations on design choices of LMMs, we refer to several recent studies [51, 63, 64, 104, 133, 10].”
TinyVLA: Toward Fast, Data-Efficient Vision-Language-Action Models for Robotic Manipulation (2024, neutral): “Theﬁrsttwowereusedinthe mainexperiments,whileTinyVLA-3Butilizesthepre-trained PaliGemmamodel[45].”
OCRBench: on the hidden mystery of OCR in large multimodal models (2023, neutral): “[64] Shanghai AI Lab 842 PaliGemma-3B-mix-448 [65] Google 614 CongRong [66] CloudWalk 827 Cambrian-13B [63] NYU 610 GLM-4v [67] Zhipu AI 814 MiniCPM-V-2 [68] OpenBMB 605 MiniMonkey-2B [69] HUST 802 Cambrian-34B [63] NYU 591 InternVL2-8B [64] Shanghai AI Lab 794 CogVLM-17B-Chat [67] Zhipu AI 590…”
SmolVLA: A Vision-Language-Action Model for Affordable and Efficient Robotics (2025, neutral): “We also compare against two variants of π 0 : one initialized from a vision-language model (Paligemma-3B), and another further pretrained on robotics datasets (intitialized from the weights released by the authors).”
LightSTAR: Efficient Visual Document Retrieval via Lightweight Selection with Vision-Adaptive Refinement (2026, unknown): “The emergence of powerful generalist MLLMs—such as LLaVA [33], PaliGemma [3], Qwen-VL [2], and InternVL [6]—has further advanced document understanding capabilities.”
Nemotron ColEmbed V2: Top-Performing Late Interaction embedding models for Visual Document Retrieval (2026, unknown): “Notable implementations include ColPali [7] (PaliGemma-based [26]), ColQwen [7] (Qwen2-based [27]), Jina Embedding Models [28] (Qwen2.”