AI BriefingTII Releases Falcon 180B, a 180B-Scale Model
Key point
TII has released Falcon 180B, a large-scale open-source LLM with 180 billion parameters.
Details
TII (Technology Innovation Institute) has released Falcon 180B, a large language model (LLM) with 180 billion parameters, via Hugging Face. This is one of the largest open models currently available, trained using 3.5 trillion tokens.
Falcon 180B was trained on a vast amount of data centered around the RefinedWeb dataset, including conversations, technical papers, and code. The training process involved up to 4,096 GPUs running simultaneously via Amazon SageMaker, taking a total of 7 million GPU hours.
In terms of performance, Falcon 180B outperforms Llama 2 70B and OpenAI's GPT-3.5 on major benchmarks such as MMLU, showing performance comparable to Google's PaLM 2-Large.
Key Features and Notes:
- Model Configuration: Both a Base model and a Chat model are provided.
- License: Commercial use is allowed, but very restrictive conditions apply excluding 'hosting use,' so legal review is required.
- Architecture: Extends Multi-query attention, an innovation from Falcon 40B, to enhance scalability.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.
Related
Qwen3.5-based Small Decision Model 'Kev' Released with TypeSafe API Compatibility and Performance Metrics