AI BriefingTII Releases Falcon 2 11B Model and VLM
Key point
TII has released the 11B-scale Falcon 2 LLM and VLM, trained on over 5 trillion tokens.
Details
TII (Technology Innovation Institute) has launched Falcon 2, its next-generation model series for the open source community. This release focuses on increasing usability through lower inference costs and multimodal support.
The Falcon 2 11B LLM was trained on a total of over 5,000B (5 trillion) tokens, including the high-quality filtered dataset RefinedWeb. Through a 4-stage training strategy, the context length was extended up to 8,192 tokens, and the model supports 10 languages including English, Spanish, French, and German.
Also released alongside it is an 11B VLM (Vision-Language Model) with integrated image understanding capabilities. This allows users to converse about visual content using text.
The model has an architecture composed of 60 Transformer blocks, and was trained using 1,024 A100 40GB GPUs with a 3D parallelism strategy (TP=8, PP=1, DP=128).
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.