AI BriefingopenPangu-2.0-Flash Model Released
·2026.07.01 19:27
Key point
openPangu-2.0-Flash, a 92B MoE model based on Ascend, has been released.
Details
openPangu-2.0-Flash is a MoE (Mixture-of-Experts) structured model trained on Ascend hardware, with 92B total parameters and 6B active parameters. It supports a long context length of 512k and was pretrained on a total of 34T tokens of data.
The key technical features are as follows:
- Efficient attention structure: While maintaining MLA (Multi-head Latent Attention), it optimizes computation and memory usage by combining DSA (Dense Sparse Attention) and SWA (Sliding Window Attention) in a 1:2 ratio.
- Improved residual topology: The existing residual path was replaced with a 4-stream mHC design to enhance representational diversity and generalization ability.
- Inference acceleration: It supports Self-speculative decoding, using 3 MTP (Multi-token Prediction) heads to generate 3 additional tokens step by step.
- Optimization algorithm: The Muon optimizer was used for fast convergence.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.
Related
Stanford and NVIDIA Release 'Contrastive Language Model' CLM with 9x Faster Inference
Meta Introduces Muse Realtime Avatar for Synchronized, Expressive Video Generation'Cache-to-Cache' Framework Revealed for Direct KV-cache Communication Between LLMs Without Text