AI Briefing#llm-training
The latest AI and developer news about #llm-training, with the original source and a short summary.
The latest AI and developer news about #llm-training, with the original source and a short summary.
Hugging Face has released The Stack v3, the largest open-source code dataset to date.
A Huawei-led consortium released a technical report demonstrating the training performance of DeepSeek V4 models using Huawei Ascend chips.
Prime Intellect has launched verifiers v1, an environment stack for agentic RL and evals.
Meta is expanding Hyperion, a $50 billion-scale mega AI infrastructure project in Louisiana.
An open-source kernel has been released that enables efficient training of MiniMax Sparse Attention (MSA) on Hopper and Blackwell GPUs.
Large-scale scraping to collect training data for LLMs is threatening the web ecosystem.
Analyzes the memory spike problem that occurs during Cross Entropy calculation in LLM training and presents solutions.
Meta is forcibly restructuring its engineering organization into a data-labeling-focused ADO organization to secure AI competitiveness, triggering internal conflict.
Recursive unveiled early results from an automated AI research system that achieved SOTA in model training and GPU kernel optimization.
Hugging Face has released Open-R1, an open-source project to reproduce DeepSeek-R1's training pipeline.
The autonomous research agent Aiden outperformed human participants to record the most contributions in OpenAI's Parameter Golf challenge.
Introducing On-policy distillation (OPD), a core training technique used in the latest models such as Qwen and DeepSeek.
By applying DPO to OCR tasks, text degeneration that is hard to resolve with SFT alone was significantly improved.
Unsloth Studio now supports model training using MLX on Mac.
An AI-contamination-free dataset of 103B tokens built from Usenet data spanning 1980 to 2013 has been released.
TRL library added a Delta Weight Sync feature that transfers only the changed weights, dramatically reducing bandwidth consumption during RL training.
TIME methodology and TIMEBench have been released, training Qwen models to perform short reasoning only when needed.
Unsloth collaborated with NVIDIA to boost LLM training speed by about 25%.
Meta and Zuckerberg have been sued over alleged copyright infringement in training Llama.
AutoSP is a DeepCompile-based compiler that automates Sequence Parallelism for long-context training.
Built on Simula research, OpenSimula has been released to control the diversity of synthetic data.
An analysis of the design patterns and technical differences among 16 open-source libraries aimed at improving the efficiency of Asynchronous RL.
Hugging Face has integrated Ulysses Sequence Parallelism, which supports long-context training of over a million tokens, into its major libraries.
It covers the technical challenges and solutions involved in applying Agentic RL to the GPT-OSS model to optimize agent performance.