Miles: A PyTorch-Native Stack for Large-Scale LLM RL Post-Training

As reinforcement learning (RL) takes on a larger role in LLM post-training, and as model scale grows along with the adoption of MoE (Mixture-of-Experts) architectures, RL is evolving beyond a simple training loop into a complex distributed systems problem.

Miles is an open-source framework from RadixArk designed to address these challenges. The framework combines SGLang for high-performance sample generation, Megatron-LM for scalable distributed training, Ray for cluster orchestration, and PyTorch as the core numerical computation layer.

Miles' architecture follows a "small-core, many-edges" philosophy. It keeps the core training loop concise while allowing users to easily swap out rollout logic, reward computation, loss functions, and more as Python modules, improving customization convenience for both researchers and infrastructure teams.

Key features include:

• High-performance rollout and training: Integration of SGLang and Megatron-LM resolves bottlenecks between generation and training
• MoE optimization: Supports matching routing behavior between the rollout and training stages
• Efficient resource management: Distributed system orchestration and GPU-aware scheduling using Ray
• Reliability and observability: Built-in fault tolerance, checkpointing, and real-time observability features