AI Briefing#gpu-scheduling
The latest AI and developer news about #gpu-scheduling, with the original source and a short summary.
The latest AI and developer news about #gpu-scheduling, with the original source and a short summary.
In Kubernetes 1.34, DRA transitioned to GA, allowing flexible, attribute-based requests for GPUs.
A new scheduling strategy keeps workloads within NVLink domains to avoid 43% performance loss, yielding 27% more usable GPUs and 14.4% higher throughput in pre-training.
Introducing GPU time budgets and fair-share allocation instead of priorities significantly improves wait times while maintaining 98% cluster utilization
The new system reduced p90 debug workload wait times from 2 hours to 30 seconds and cut on-call maintenance toil by 74% across clusters of thousands of NVIDIA GPUs.
AI21 eliminated 20 weekly manual interventions and reduced hero job starvation from 72 to 12 hours by implementing Kueue on a 10,000-GPU cluster.
A constraint-based GPU allocation method improved GPU utilization by 33 percentage points compared to the existing FIFO method.
This introduces a technology that maximizes infrastructure efficiency by reallocating idle GPU resources—caused by traffic fluctuations in LLM services—to model training and research.
We built an efficient scheduling pipeline that reallocates idle GPU resources, which occur due to traffic fluctuations in LLM services, for research and training purposes.
LG AI Research leveraged vLLM metrics to repurpose idle GPUs for research jobs, maximizing infrastructure efficiency.