Group Sequence Policy Optimization

By Chujie Zheng, Shixuan Liu, Mingze Li, Xiong-Hui Chen, Bowen Yu, Chang Gao, Kai Dang, Yuqiong Liu, Rui Men, An Yang and 2 more. arXiv.org, 2025

600 citations recorded by the source corpus.

This paper introduces Group Sequence Policy Optimization (GSPO), our stable, efficient, and performant reinforcement learning algorithm for training large language models. Unlike previous algorithms that adopt token-level importance ratios, GSPO defines the importance ratio based on sequence likelihood and performs sequence-level clipping, rewarding, and optimization. We demonstrate that GSPO achieves superior training efficiency and performance compared to the GRPO algorithm, notably stabilizes Mixture-of-Experts (MoE) RL training, and has the potential for simplifying the design of RL infrastructure. These merits of GSPO have contributed to the remarkable improvements in the latest Qwen3 models.

View Paper: https://www.semanticscholar.org/paper/a0b04def806f1d5d0126f98334b664cc57a42a0d

Citations in the corpus

• Qwen3-Omni Technical Report (2025, unknown): “Finally, we leverage GSPO (Zheng et al., 2025) to comprehensively enhance the model’s capabilities and stability across various modalities, including text, image, video, and audio.”
• InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiency (2025, unknown): “During the online RL stage, we employ GSPO [183], without reference model constraints, as our online RL algorithm, which we find more effective in training both dense and mixture-of-experts (MoE) models.”