Reflection releases Beam: a 501B MoE open model with 1M context and specialized agentic capabilities

Reflection, a new US research lab, has released Beam, a 501B parameter Mixture-of-Experts (MoE) large language model under the Apache 2.0 license. The model is optimized for coding and agentic use cases, featuring 23B active parameters per token and a 1 million token context window.

Architecture and Efficiency

Beam utilizes a hybrid attention mechanism to optimize memory usage. It consists of 52 layers, with 43 sliding-window layers (512-token window) and 9 global layers (full context). This design reduces KV cache growth to approximately 1/6 of naive methods; for a 1M context, the KV cache requires 36.08 GiB compared to 208 GiB for standard implementations.

The MoE structure includes 256 experts across 49 layers, accounting for 98.4% of the model's parameters. The router follows a DeepSeek-V3 group-limited scheme, selecting the top 8 experts from 4 selected groups. Global layers use scalable-softmax query scaling instead of positional encoding to maintain precision over long contexts.

Deployment and Serving

Beam is available in BF16 (~1 TB), FP8 (for Hopper/Ampere GPUs), and NVFP4 (~280 GB, for NVIDIA Blackwell GPUs). Day-0 support is provided for transformers, SGLang, and vLLM.

• Hardware Requirements: NVFP4 checkpoints require 4 B200 GPUs (720 GB) or 2 B300 GPUs. FP8 checkpoints can run on 8 H200 GPUs.
• Speculative Decoding: vLLM supports the reflection/dflash2-drafter-preview model, allowing up to 15 tokens to be predicted per forward pass.
• Inference Providers: Available via Baseten, Fireworks, and Together through Hugging Face Inference Providers.

Benchmarks and Performance

Reflection reports strong performance on agentic and reasoning benchmarks:

• SWE Bench Pro v2-Hard: 78.6% (NVFP4), 78.4% (FP8), 76.5% (BF16).
• Terminal Bench v2.1: 80.1% (BF16).
• AIME 2026: 97.7% (BF16).
• GPQA Diamond: 90.6% (BF16).
• MCP Atlas: 80.8% (BF16).

Post-Training and Fine-Tuning

The release includes recipes for post-training using TRL and Miles (Reflection & RadixArk).

• SFT: Fine-tuning with LoRA adapters on the Multilingual-Thinking dataset improved language adherence from 3/16 to 16/16 requests.
• GRPO: Full-parameter GRPO training on 96 H200 GPUs improved AIME 2025 accuracy from 46.25% to 55.8%. LoRA GRPO on 32 H200 GPUs improved accuracy from 42.9% to 64.2% while reducing mean output length.

Ecosystem and Tools

• Transformers: Requires version >=5.20.1. Supports parse_response for separating reasoning, answers, and tool calls, and reasoning_effort parameters (low to max).
• Demos: Includes examples like BeamSpeak (macOS dictation app) and Decision Model Fine-tuning, where a Qwen3.5-0.8B model fine-tuned with Beam's guidance achieved 76.25% accuracy in incident triage versus 4.69% for zero-shot prompting, with an 11.9x latency speedup.