NVIDIA put a memory size in the model's name. Nemotron 3.5 Bolt 74B-A9B 24GB went up on Hugging Face on 15 October: a 74B mixture of experts squeezed to about 2 bits so it fits "in 24GB of memory". Then you open the llama.cpp card, and the two recipes it gives are labelled 28GB and 32GB. If you own a 24 GB RTX 4090 and you'd planned your weekend around this one, read that table first.
The short answer
Bolt is 74B total and 9B active, with ungated weights under OpenMDW-1.1, a permissive licence that allows commercial use. The official GGUF is 22.02 GB, and NVIDIA's recipes also load a 1.73 GB vision projector and a 1.00 GB speculative drafter. Those recipes target RTX Spark machines with 64 GB or 128 GB, or discrete cards with 32 GB or more. There's no 24 GB discrete recipe. And the launch post's "#1 open model in its class on PinchBench" sits next to a model card table where Qwen3.6-35B-A3B scores 88.07 to Bolt's 87.82.
Where the 24 GB went
The name describes the packed model, not your card. NVIDIA's card says the architecture search "targets a deployment footprint below 24 GB at batch size 1 and 128K context". Every expert, routed and shared, is stored as IQ2_XXS at 2.0625 bits per weight; the Mamba layers stay in NVFP4 and the embedding and output layers in Q8_0. That's a serious piece of engineering. It's also aimed at one machine. The card says the experts were quantized "to fit the network in RTX Spark's memory", and NVIDIA's post says RTX Spark laptops arrive on 16 October.
Now the arithmetic, which is ours, from the repo file lists. The three files the llama.cpp recipe loads weigh 22.02 + 1.73 + 1.00 = 24.75 GB in decimal units. That's about 23.05 GiB. A card sold as 24 GB holds 24 GiB, so you'd have less than 1 GiB left for the KV cache, the Mamba state, the compute buffers and whatever your desktop already keeps on the GPU. NVIDIA's smaller recipe, the one labelled 28GB, runs a 65,536-token context with a q8_0 KV cache and asks for a 32 GB discrete card or a 64 GB Spark. The 32GB recipe gets the full 262,144 tokens and wants 48 GB or the 128 GB Spark.
Could a 4090 run it if you drop the vision projector and the drafter and cut the context? Probably, at some context length. We haven't tried, NVIDIA doesn't publish that setup, and we won't guess a number. Unsloth's GGUF repo has 1-bit builds at 20.73 GB (IQ1_S) and 22.52 GB (IQ1_M) that leave more headroom. Nobody has published scores for them, though, and every number on NVIDIA's card was taken on the official IQ2 build.
The vLLM route is narrower still. It wants a pinned nightly container plus the b12x kernels, and the card lists Blackwell SM120 and SM121 as the supported microarchitectures, even though its own summary table mentions A100 and H100. I'd treat that summary line as hopeful until someone reports a run on Hopper.
The PinchBench line, and the rest of the table
NVIDIA's launch post says Bolt "is the #1 open model in its class on PinchBench". The model card, published the same day, gives Qwen3.6-35B-A3B 88.07 and Bolt 87.82. A quarter of a point is inside the noise of most agent harnesses, so it's not a scandal. It's still the wrong way round for a #1 claim, unless "its class" quietly leaves out the model in the first column. The card also mentions PinchBench scores "up-to 90.4" over ten runs from the RL stage. That isn't the number in the table, and the card doesn't say it's the build you download.
Where Bolt leads its own table, it leads by a lot. BrowseComp is 66.75 against Qwen's 48.74. Terminal-Bench 2.1 is 50.98 against 44.38. Keep the setup in mind, though: the rivals run in BF16 at roughly three times Bolt's memory, and NVIDIA ran all of them. Bolt also loses rows. Gemma4 26B-A4B beats it on AA-Omniscience accuracy (22.17 to 16.90), and NVIDIA's own Lightning 30B beats it on non-hallucination (68.81 to 61.27).
One paragraph deserves a slow read before you trust the agent scores. The training section says the text fine-tuning blend used synthetic data generated by Kimi-K3 "targeting" AIME, GPQA Diamond, Terminal Bench 2.1, SciCode, AA-LCR, APEX and LMArena, plus a small set of GLM 5.2 trajectories aimed at MCP-Atlas. NVIDIA is unusually open about this, and we'd rather labs said it than hid it. It still means several rows in that table measure a model trained toward them.
Should you pull it?
If you've got an RTX Spark on order, or a 32 GB card like the RTX 5090, it's the most interesting local agent model this month. The weights are ungated, the vision tower comes with them, and OpenMDW-1.1 lets you use and redistribute it commercially as long as you keep the licence and notices; your rights end if you sue someone over the model. On a 24 GB card, wait until someone posts a working context length, or try the 1-bit builds knowing nobody's scored them. I might be wrong about how much context fits, and that's exactly why we'd rather wait for a real report than quote one.
Super VL, the 120B-A12B parent it was compressed from, is a different animal: its card names 4× H100 or 2× GB200, and the NVFP4 repo alone is 75.06 GB. The small Bolt E4B is due on 23 October. Both released models are now on our open-weight models tracker, next to Nemotron 3.5 Lightning, the smaller August release that's 21.6 GB in NVFP4.
FAQ
Does Nemotron 3.5 Bolt run on a 24 GB GPU?
NVIDIA doesn't publish a recipe for one. Its llama.cpp recipes are labelled 28GB and 32GB and ask for a 32 GB or 48 GB discrete card, or an RTX Spark with 64 GB or 128 GB. The three files those recipes load total 24.75 GB, about 23.05 GiB, before any KV cache.
What licence is Nemotron 3.5 Bolt under?
OpenMDW-1.1. It allows commercial use, modification and redistribution, asks you to keep the licence and notices when you redistribute, and ends your rights if you start patent or copyright litigation over the model.
How big are the Bolt files?
The official GGUF is 22.02 GB, the vision projector 1.73 GB and the DFlash drafter 1.00 GB. Unsloth also offers 1-bit builds at 20.73 GB and 22.52 GB, with no published benchmark scores.
Is Nemotron 3.5 Bolt E4B out?
Not yet. NVIDIA's post says Super VL and Bolt 74B-A9B are available now and Bolt E4B is coming on 23 October.
Sources: NVIDIA, Nemotron 3.5 Super VL and Bolt announcement (15 October 2026); Hugging Face model cards for Bolt 74B-A9B 24GB, its GGUF build, the LiLiCorr drafter and Super VL 120B-A12B; the OpenMDW-1.1 licence; independent: Unsloth's Bolt GGUF repo and Unsloth's release note. File sizes read from the Hugging Face API on 15 October 2026.
