AI Briefing1T-Parameter-Class Multimodal Model Inkling Released
·2026.07.15 09:00
Key point
Thinking Machines has unveiled Inkling, a large-scale multimodal MoE model with 1 trillion parameters.
1 / 2
Details
Thinking Machines has released Inkling, an open model with 1T (1 trillion) parameters supporting text, images, and audio, on Hugging Face. The model was trained on 45 trillion tokens and supports a 1M (1 million) context window.
Key technical features:
- MoE (Mixture-of-Experts) architecture: Of the total 975B parameters, only 41B parameters are activated during inference, improving efficiency, and the model has 256 experts.
- Hybrid Attention: Global attention and sliding window attention are interleaved at a 5:1 ratio to maximize computational efficiency.
- Relative Attention: Uses relative attention instead of RoPE to encode positional information.
- SConv (Short Convolution): Utilizes 1D convolution to aid local representation learning.
- Multimodal support: Processes images and audio through a simplified hierarchical MLP patchifier instead of a separate encoder, and also includes video processing capabilities.
It is currently available for immediate use in transformers, SGLang, and llama.cpp, with both BF16 and NVFP4 quantized versions provided.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.
Related
New US-made AI Model Designed for Customization