SkyPilot Topology-Aware Scheduler Boosts NVL72 Efficiency by 27% via Rack-Aware Placement

• NVLink and InfiniBand performance gap causes 43% degradation for distributed training jobs crossing rack boundaries, necessitating rack-aware scheduling for NVL72 systems.
• NVIDIA modeling shows 16-node slices provide 27% more usable capacity than 18-node slices in a 2,304-node cluster, assuming 2% node unavailability.
• Rack-aware slice-based scheduling increased throughput by 14.4% and reduced iteration time from 1.59s to 1.39s during GR00T N1.7 pre-training on 512 GB300 GPUs.
• SkyPilot Topology-Aware Scheduler (TAS) colocates multi-node jobs on single racks and uses spare nodes for hot-swapping to maintain NVLink domain integrity.

Performance Impact

• InfiniBand cross-rack traffic reduces performance by 43% compared to NVLink for the same distributed training job.
• 64-way Expert Parallelism (EP) on standard 8-GPU servers forces InfiniBand usage, potentially consuming 60% of training time in communication.
• GB200 NVL72 32-way EP achieves up to 4.4x higher per-GPU throughput for DeepSeek-R1 serving compared to 8-GPU B200 servers.

Scheduling Mechanism

• TAS splits jobs into 16-node slices (configurable) placed on separate racks with two reserved spare nodes.
• Frontier Trainer hot-swaps faulty nodes with spares within the same rack; reschedules to healthy racks if spares are exhausted.
• Automatic Defragmentation preempts safely restartable workloads to free entire racks for blocked large jobs, avoiding fragmentation in shared clusters.

Reliability and Cost

• NVL72 rack Mean Time Between Incidents (MTBI) is approximately 27 days, compared to 490 days for 4-GPU nodes as measured by Meta on its supercluster.
• On a 10,000-GPU cluster, the 14.4% speedup saves approximately $16.7K per training hour and $400K per training day.