AI BriefingGRPO Experiments with 3 Mac Minis
Key point
Trained a Reddit summarization model with GRPO on a 3x Mac Mini cluster, and quality+length reward achieved the best performance.
Details
Trained LFM-2.5-350M on Reddit post summarization, with the goal of verifying whether a small LLM could produce summaries within a 64-token length limit.
Training was conducted on a 3x Mac Mini cluster. One handled training, and two performed rollouts with vLLM-metal, using the SyncPS architecture from smolcluster.
Two reward designs were compared.
- length-only: uses only a length penalty
- length-quality: combines a length penalty with quality rewards such as ROUGE-L + METEOR
Evaluation was conducted with a DeepEval-based LLM-as-a-Judge (gpt-5) pipeline. The axes were Faithfulness, Coverage, Conciseness, Clarity, measured on 200 test samples from the smoltldr dataset.
The results showed that the setup with quality reward was superior.
- Best configuration: length-quality-meteor-rouge
- Composite 2.769
- Faithfulness 0.832, Coverage 0.511, Conciseness 0.659, Clarity 0.767
- Pass Rate 44.3%
In contrast, the length-only baseline was lower at Composite 2.416, with quality and accuracy metrics generally falling behind.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.
Related
Toss Doctor V2 Released: Maximizing QA Efficiency with LLM-based Automated Testing and Self-healing