Upstage Unveils Solar Open 2 250B

Upstage has released the Solar Open 2 (250B-A15B) open-weight model. With an MoE architecture that activates only 15B parameters per token, it delivers large-model performance at small-model inference cost.

Key features:

• Hybrid-Attention MoE: Interleaves 3 linear attention layers for every 1 softmax attention layer (KV cache is maintained for only 12 of the total 48 layers)
• 1M token context: Linear attention implicitly encodes positional information in its recurrent state, eliminating RoPE (NoPE), with no theoretical upper limit on context
• Efficient training: Only 2.3% of Solar Open 1 (102B) weights that survive the architecture change were transferred, with the rest randomly initialized, accelerating early convergence
• Multilingual: Supports English, Korean, and Japanese
• Agentic specialization: Targets tool calling, multi-step reasoning, office productivity, document work, and coding