AI Briefing#grpo
The latest AI and developer news about #grpo, with the original source and a short summary.
The latest AI and developer news about #grpo, with the original source and a short summary.
Trained Qwen2.5-0.5B-Instruct on a Reddit summarization task using GRPO.
P-GRPO reduces GRPO's bias by decoupling individual preferences using per-group reward history.
A 32-bit overflow in vLLM's Mamba CUDA kernel caused logprob mismatches.
Tuning and autoscaling vLLM for bursty traffic reduced throughput bottlenecks and latency.
By temporarily putting overly easy examples to sleep in GRPO, this approach reduces wasted computation and achieves up to 3X efficiency.
Based on Qwen3-4B, DeepMath uses Python code to improve math reasoning accuracy while cutting output length by 66%.
AIME benchmark performance was improved through high learning rates in the SFT stage and relaxed clipping strategies in RL.
The Kimina-Prover-RL pipeline, which performs Lean 4 theorem proving using DeepSeek-R1-style reasoning, has been released along with two small models.
HuggingFace's TRL library now supports the latest alignment techniques such as MPO and GRPO for VLMs.
GSPO reduces GRPO's instability through sequence-level optimization, enabling RL scaling.
TRL has integrated vLLM into the training process, reducing GPU resource waste and improving GRPO training efficiency.
Liger Kernel has been integrated into TRL, reducing memory usage by 40% during GRPO training.
PipelineRL has been released, resolving the trade-off between inference throughput and data quality during LLM reinforcement learning.
HuggingFace announced the first update on the Open-R1 project, which aims to reproduce DeepSeek-R1's training pipeline and dataset.
It explains how to reproduce DeepSeek R1's reasoning capability in a small model through the GRPO algorithm.
Qwen has released Qwen2-Math, a series of math-specialized LLMs that maximize mathematical problem-solving ability.