IBM Releases Granite Switch 4.1 Models Combining Chat, Decisions, and Speech on Single Endpoint

IBM has released Granite Switch 4.1 models (Granite-Switch-4.1-8B and -30B), which combine natural-language responses, specialized decision-making, and speech-to-text capabilities into a single hosted endpoint. This release builds on the modular Granite Switch project by integrating three key adapters into the base model without significantly increasing VRAM requirements.

Key Components

The 4.1 models include:

• A new Decision aLoRA for general-purpose decision-making.
• A specialized Guardian aLoRA that reportedly outperforms Jev 1.13, Microsoft-Decision-1, GPT-Luna-Decisions, and Clef-Flash on safety-related decision tasks.
• Granite-Speech-5.0-Turbo-CTC, identified as the fastest ASR model on the OpenASR Leaderboard, embedded for transcription tasks.

Performance and Efficiency

The models utilize Activated LoRAs (aLoRAs) that are cache-efficient, allowing the KV cache from base model generations to be reused for subsequent adapter-based decisions. This architecture, combined with recent vLLM contributions for low-rank adapter inferencing, enables rapid switching between chat, decision, and spoken requests.

In a comparison against Jev-27B, the Granite-Switch-4.1-30B completed a real-world, 5-step RAG workflow significantly faster. Additionally, a custom 3B parameter Granite Switch model demonstrated decision-making speeds of 60–100 decisions per second in a Doom game environment, compared to Jev’s reported 10 decisions per second.