IBM has announced the release of Granite-Switch-4.1-8B and -30B models, integrating multiple AI functionalities into a single endpoint. This new architecture allows for the simultaneous execution of chat, decision-making, and Automatic Speech Recognition (ASR) tasks, while maintaining VRAM requirements at the level of a single base model.
Technical Composition and Performance
The Granite Switch 4.1 models utilize Activated LoRAs (aLoRAs) technology to compose different capabilities:
- Decision aLoRA: For general-purpose decision-making.
- Guardian aLoRA: A specialized safety layer that outperforms competitors such as Jev 1.13 and Microsoft-Decision-1 on safety-related tasks.
- Granite-Speech-5.0-Turbo-CTC: The fastest ASR model on the OpenASR Leaderboard, embedded for immediate transcription.
Because these Switch adapters are cache-efficient, the KV cache from the base model generation can be reused for any subsequent adapter-based decisions, ensuring blazing-fast switching between different request types.
Specialization and Customizability
IBM emphasizes that specialized adapters from the Granite Libraries can outperform generalist decision models. In real-world RAG workflow tests, the Granite-Switch-4.1-30B completed tasks significantly faster than the Jev-27B. Furthermore, a custom 3B parameter model successfully navigated the game Doom at 60-100 decisions per second, while simultaneously narrating and responding to user questions.
Users can compose their own Granite Switch models by mixing and matching adapters from Granite Libraries or by using the upcoming Adapter SDK to train custom solutions for specialized language or classification tasks.
