Ultra-fast Inference Engine Atlas Released as Open Source

Atlas is a high-performance inference engine built with Pure Rust and CUDA, without PyTorch or a Python runtime. By eliminating the bottlenecks that occur in existing Python-based stacks, it achieves 111 tok/s (sustained) on the Qwen3.5-35B model in a DGX Spark (GB10) environment, showing performance about 3x or more compared to vLLM.

Key technical features are as follows:

• Blackwell SM120/121-specific kernels: Uses hand-tuned CUDA kernels for Attention, MoE, GDN, and Mamba-2, delivering optimized performance without generic fallbacks.
• Native data type support: Supports NVFP4 and FP8 directly on tensor cores.
• MTP (Multi-Token Prediction): Increases decoding throughput by up to 3x through Speculative Decoding.
• High compatibility: Supports OpenAI and Anthropic API specifications, enabling immediate integration with Claude Code, Cline, Open WebUI, and more.

The development team also plans future ports to AMD Strix Halo and RTX 6000 Pro Blackwell.