Gemma 3 Technical Report

By Gemma Team Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ram'e, Morgane Rivière, Louis Rouillard and 200 more. arXiv.org, 2025

1,621 citations recorded by the source corpus.

We introduce Gemma 3, a multimodal addition to the Gemma family of lightweight open models, ranging in scale from 1 to 27 billion parameters. This version introduces vision understanding abilities, a wider coverage of languages and longer context - at least 128K tokens. We also change the architecture of the model to reduce the KV-cache memory that tends to explode with long context. This is achieved by increasing the ratio of local to global attention layers, and keeping the span on local attention short. The Gemma 3 models are trained with distillation and achieve superior performance to Gemma 2 for both pre-trained and instruction finetuned versions. In particular, our novel post-training recipe significantly improves the math, chat, instruction-following and multilingual abilities, making Gemma3-4B-IT competitive with Gemma2-27B-IT and Gemma3-27B-IT comparable to Gemini-1.5-Pro across benchmarks. We release all our models to the community.

View Paper: https://www.semanticscholar.org/paper/adac25c0aebd013f1225ec02636edfcc0bf4cf7c

Citations in the corpus

• InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiency (2025, unknown): “5 shows broad improvements, and the 14B variant surpasses larger counterparts such as Gemma-3-27B [122].”
• GLM-4.5V and GLM-4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learning (2025, unknown): “1V-9B-Thinking against a wide range of open-source state-of-the-art MLLMs, including Step-3 [52], Qwen-VL series [4], Kimi-VL [50] and Gemma-3 [48].”
• OntoLearner: A Modular Python Library for Ontology Learning with Large Language Models (2026, unknown): “Similarly, from the Gemma family (Kamath et al. 2025), we included instruction-tuned models at varying scales, namely Gemma-3-1b-it, Gemma-3-4b-it, Gemma-3-12b-it, and Gemma-3-27b-it.”
• ER-MIA: Black-Box Adversarial Memory Injection Attacks on Long-Term Memory-Augmented Large Language Models (2026, unknown): “2:3b as the victim LLM, with additional experiments on Gemma 3:27b (Team et al., 2025) and GPT-oss:20b (Agarwal et al., 2025).”
• Benchmarking Small Language Models and Small Reasoning Language Models on System Log Severity Classification (2026, unknown): “Gemma3-4B and Gemma3-1B also use a modern Transformer stack with GQA and QK-Norm, but their attention is organized into interleaved local and global blocks tailored for long-context throughput (e.g., 5 local layers followed by 1 global layer) [85].”
• Knowledge Enhanced Efficient Robotic Decision Making with Intuitive-Deliberate LLMs (2025, unknown): “All language model operations throughout implementation utilized Google’s Gemma-3-27B with Q4_K_M quantization [48], configured with a temperature setting of 0.1 for consistent outputs.”