AI BriefingRequest to Reproduce Early LLM Training Results
Key point
A 1.2B LLM trained with a new training signal achieved 59.9% preference in a blind evaluation.
Details
A 1.2B parameter LLM applying two new loss shaping techniques showed an advantage in blind preference evaluation.
-
per-token gain weighted each token loss according to surprisal while maintaining the average gradient budget.
-
per-layer divergence scaling adjusted the gradient according to how much each transformer block changed the representation in the forward pass, with the overall scale normalized.
-
The comparison group used standard cross-entropy, and both models were trained on the same data, same order, and same seed up to 30,000 steps, 3.9B tokens.
The smoothed validation loss was statistically the same between the two models. However, in 1,181 pairwise judgments left by 42 blind evaluators (29 humans, 13 foundation-model judges), the gain-trained model achieved 59.9% of 784 decisive comparisons, with 33.6% ties.
The decisive preference rates for humans and foundation-model judges were nearly identical at 60.5% and 59.0%, respectively. Of 32 questions, the human majority and FM majority agreed in direction on 26, and the per-question gain rate correlation was Pearson r = 0.78. Under the assumption of per-judgment independence, a two-sided binomial p = 2.80e-8 was presented.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.
Related
Toss Doctor V2 Released: Maximizing QA Efficiency with LLM-based Automated Testing and Self-healing