AI BriefingGrok-1 Open Source Release
·2024.03.17 00:00
Key point
xAI has released the weights and architecture of **Grok-1**, a MoE model with 314 billion parameters, under the Apache 2.0 license.
Details
xAI has open-sourced the base model weights and network architecture of its large language model (LLM), Grok-1.
Grok-1 is a Mixture-of-Experts (MoE) model with 314B (314 billion) parameters, with 25% of the total weights activated per token.
Key details are as follows:
- It is a Raw Base Model checkpoint that completed pretraining in October 2023, and has not been fine-tuned for specific purposes such as conversational services.
- It was trained from scratch by xAI using a custom training stack based on JAX and Rust.
- It is distributed under the Apache 2.0 license.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.
Related
Foremerge v0.5.0 Released: Rust-Based Open-Source Tool Detects Intent Conflicts Between Parallel AI Coding AgentsQwen3.5-based Small Decision Model 'Kev' Released with TypeSafe API Compatibility and Performance Metrics