numbat: Checkpoints of Anthropic's Unreleased 14T Parameter Model Revealed

A mid-training checkpoint of Anthropic's next-generation model, codenamed 'numbat', identified in the Claude Code source code leak, has been released. It is a large language model with a total of 14 trillion parameters, featuring a hybrid GDN/MLA architecture and a MoE structure with 1,224 experts.

The model is stored in NVFP4 quantization format, divided into 772 shards, and occupies a total capacity of 7.72TB. The repository includes the model configuration files along with complete reference implementation code, allowing tensors to be loaded and inference to be executed via standard libraries.

This checkpoint is the original output of the pretraining stage, having progressed to 1.83 million steps out of approximately 1.96 million steps. Since post-training or RLHF processes have not been applied, it is more suitable for large-scale model structure analysis or research purposes than for general interactive AI services.

A reference implementation that operates on a single device is provided, but actual inference requires a significant amount of GPU memory and storage space. It is a useful resource for developers who want to directly examine the internal structure and training logs of an unreleased model.