Granite Switch 4.1 won’t install next to this month’s vLLM

One checkpoint, three jobs. That's the pitch for IBM's Granite Switch 4.1, which landed on Hugging Face on 15 October: a Granite 4.1 model that chats, scores multiple-choice decisions, flags unsafe prompts and transcribes speech, all behind one vLLM endpoint. It's Apache 2.0 and ungated. The catch isn't the licence. It's the install line. If you upgraded vLLM or Transformers in the last few weeks, this model won't share that environment with you.

What IBM shipped, from the repos

Two production checkpoints went up on 14 October and were announced the next day: granite-switch-4.1-8b and granite-switch-4.1-30b. We summed the file lists through the Hugging Face API. The 8B repo holds 8.78B parameters in BF16 across four safetensors shards, 17.56 GB in all. The 30B repo is 29.99B parameters and 59.99 GB. There's no FP8 or GGUF build from IBM yet, so plan on BF16 sizes plus your KV cache.

Each checkpoint folds three extras into the base model. A decision adapter returns a probability over labelled options instead of free text. A guardian-core adapter screens for harm, jailbreaks, profanity, bias and the usual list. And Granite Speech 5.0 TurboCTC, a 470M-parameter recognizer, kicks in on its own when a message carries input_audio. You switch adapters by passing {"adapter_name": "decision"} inside chat_template_kwargs. No second deployment, no router in front.

The adapters are Activated LoRAs, and IBM's announcement says the composed model uses about the same VRAM as the base model alone, because the adapter passes reuse the base model's KV cache. We haven't measured that. It's plausible for small LoRA weights, but nobody outside IBM has published numbers yet, and that's the claim we'd want checked before we'd retire a separate safety classifier. The preview repos, which date back to 1 May and were refreshed on 15 October, are a different bundle: more adapters (factuality detection, factuality correction, policy guardrails) and no audio at all. A 3B exists only as a preview.

The version pins nobody mentions

Here's what the announcement doesn't say. The model cards list the backend requirements as vLLM ">= 0.26.0, < 0.28.0" and Transformers ">= 5.16, <= 5.17.0". The install step clones IBM's granite-switch repo on GitHub and runs pip install -e ".[vllm]", which registers a vLLM plugin. Open that repo's pyproject.toml and the vLLM range is wider: ">=0.26.0,<0.31.0", with a comment saying every line from 0.26 through 0.30 is supported by one tree. So the card and the code disagree by three minor releases.

The code is the one pip will obey, so we'd trust it over the card. It still leaves you short. PyPI shows vLLM 0.31.0 published on 5 October, and that's what a fresh pip install vllm pulls today. Transformers is tighter: 5.17.0 shipped on 9 September, and three releases have followed, up to 5.20.1 on 16 October. So if anything else in the same virtualenv needs a recent Transformers, and this month's new models tend to, the two won't coexist. The Python floor differs too: the card says 3.9 or later, the package says 3.11 to 3.13.

None of this is a scandal. Pinning a plugin to versions you've tested is responsible, and IBM's repo shows CI work on 8 October that derives GPU test runs from the pyproject. But "swap it in for your existing endpoint" reads differently once you know the endpoint has to run a vLLM from September. If your serving fleet is on 0.31, that's a separate image, not a model change.

Where it fits, and where we'd wait

The appealing case is a small team running chat plus moderation plus voice notes, today on three models. Collapsing that onto one 8B checkpoint is a real simplification, if the shared cache holds up. Mind the arithmetic, though: 17.56 GB of weights leaves little room for context on a 24 GB card, so a 48 GB card is the comfortable fit. The decision adapter is the most interesting piece to us: one token out, a probability per label, which is cheap for routing or triage.

We'd keep two doubts in view. IBM's own decision benchmark (its Decision Index, v0.3 public subset) gives the 8B 42.9 overall and the 30B 50.1, with the 8B at 18.0 on the arts slice. That's IBM grading IBM on IBM's test, and the card itself says to treat the probabilities as "a prior", not a certified classification. And the speech cascade isn't trained jointly with the language model. The card warns that a mistranscription can silently change what the model or an adapter responds to. English only, no punctuation. For a voice front end in French or German, it's not ready.

Our take: try the 8B in its own container, pinned exactly as the pyproject says, and don't fold it into your main vLLM image until IBM moves the ceiling past 0.31. We'll add a row to our open-weight models tracker and watch for that bump.

FAQ

Sources: IBM Granite, Granite Switch 4.1 announcement (15 October 2026); Hugging Face model cards for Granite Switch 4.1 8B and 30B; the granite-switch pyproject.toml on GitHub; release dates from PyPI for vLLM and Transformers; independent: RuntimeWire. File sizes read from the Hugging Face API on 17 October 2026.