AI21 Unveils Jamba 1.5 Model Family

AI21 has introduced a new open model family including Jamba 1.5 Mini and Jamba 1.5 Large. These models are built on a hybrid SSM-Transformer architecture, delivering exceptional long-context processing capability along with speed and quality that go beyond the limitations of existing Transformer models.

Key strengths include:

• 256K context window: Supports one of the longest context lengths on the market, improving quality for long document summarization, analysis, and RAG workflows.
• Speed and efficiency: Up to 2.5x faster on long context, achieving the fastest speed among comparable models.
• Proven quality: Jamba 1.5 Mini surpasses Mixtral 8x22B on the Arena Hard benchmark, while the Large model recorded higher performance than the Llama 3.1 70B and 405B models.
• Developer convenience: Natively supports JSON output, function calling, document object handling, and citation generation.

In particular, the SSM-Transformer structure combines the quality of Transformers with the efficiency of Mamba, lowering memory footprint. AI21 also developed ExpertsInt8, a new quantization technique optimized for MoE models, enabling the Jamba 1.5 Large model to run using the full 256K context on a single 8 GPU node.