Short version. Hugging Face and Google say TPU is now a normal PyTorch device: pip install torch-tpu, then device_map="tpu" in Transformers or .to("tpu") in Diffusers. The backend is version 0.2.0, published on PyPI on 19 October. The headline number, 96.5% of tests passing, only counts the tests that run on a TPU at all, and every benchmark in the post comes from Cloud TPU v6e.
If you've ever tried to move a PyTorch model onto a TPU, you know the drill: port it to torch_xla or give up and learn JAX. That's the pitch TorchTPU is meant to end. A Hugging Face post dated 20 October says TPU is now "just another device", with Transformers, Diffusers and the Kernels library running on it natively. We read the post, the repository README and Google's own write-up of the design, and the picture is good, though early.
What changes in your code
Almost nothing, which is the point. In Transformers you pass device_map="tpu" to from_pretrained and the weights load straight onto the chip. The post's example is Qwen3.5-9B. In Diffusers it's pipe.to("tpu") for eager mode, and one extra call, pipe.transformer.compile(backend="tpu", fullgraph=True, dynamic=False), for compiled mode. Models too big for one chip can be sharded with a TensorParallelConfig while they load.
Install is one line: pip install torch-tpu --extra-index-url https://download.pytorch.org/whl/cpu. The extra index matters. Without it pip pulls the much larger CUDA build of torch, which a TPU VM doesn't need. The repository lists Python 3.11 or newer, a Linux host and an Apache 2.0 licence, and its newest PyPI release is 0.2.0, uploaded on 19 October. The only other release on the index is a 0.0.0 placeholder from September 2025, so this is the first real package, and it's a day old.
What the numbers cover
Start with the 96.5%. The post says that "of the tests that run on TPU, 96.5% pass". Read the clause twice. It doesn't say what share of the whole Transformers suite runs on a TPU, and we couldn't find that figure. The team also reports 34 issues filed against TorchTPU, 19 already fixed, plus bugs found and fixed upstream in Transformers (StaticCache and the Qwen3 tensor-parallel plans). That's a healthy loop. It's also a reminder that you'd be running on a backend that's still being shaken down.
The kernel result is the loudest. With a ragged-dot kernel from the Tokamax library, loaded through the Kernels Hub, decoding at batch size 64 is 21.5 times faster on Qwen3-30B-A3B and 12.8 times faster on Gemma 4 26B-A4B, measured on a TPU v6e-8. Those are ratios against "the default implementation", and the post doesn't give absolute tokens per second in the text. A 21x gain over a default MoE path usually says the default was slow, not that the new kernel is magic. The post itself says the win grows with batch size, so a small interactive batch will see much less.
The image numbers are more modest and easier to trust. On a v6e-4, a 1024 by 1024 image with 40 steps took 3.5 seconds compiled and 8.0 seconds eager. A 1344 by 768 video clip with 49 steps took 188 seconds compiled and 262 seconds eager. Eager mode is the default, and compiled mode is what you'd deploy. The first compiled call pays the compile cost, and each later call with the same image size reuses the graph.
What we'd check before moving a workload
Hardware first. The only TPU named in the Hugging Face post is v6e, and neither that post nor the README gives a supported-generation list. If you're on another generation, test before you plan around it.
Shapes second. Google's design write-up says torch.compile goes through Dynamo, then XLA and StableHLO, not Torch Inductor, and that dynamic sequence lengths and batch sizes trigger recompilation. Bounded dynamism is on its 2026 roadmap, not in hand. If your serving traffic has ragged lengths, that's the first thing to measure. The write-up gives no general-availability date either.
Our view: for batch jobs, diffusion and prototyping on a TPU you already pay for, this is worth an afternoon now. For production serving with variable shapes, we'd wait for a few more point releases. We might be too cautious, since the issue count is dropping fast, but a 0.2.0 is a 0.2.0. If you're comparing what runs where, our open-weight models tracker lists sizes and hardware needs for the models people most often load this way.
FAQ
Do I still need torch_xla or JAX to use a TPU?
Not for the paths the post covers. TorchTPU is a native PyTorch backend, so Transformers and Diffusers code can target the "tpu" device directly. Code that already depends on torch_xla isn't automatically migrated.
Which TPUs does it support?
The Hugging Face post benchmarks on Cloud TPU v6e (v6e-4 and v6e-8) and names no other generation. The README recommends v6e GCE instances for development. Treat everything else as untested until you've run it.
Is eager or compiled mode the default?
Eager. Compiled mode needs an explicit call, and in the post's Diffusers examples it was roughly 1.4 to 2.3 times faster than eager once the graph was built.
Is 96.5% passing good enough for production?
It's a progress figure, not a guarantee. It counts only the tests that run on TPU, and the post lists known failures being filed as issues. Run your own model's tests on the exact TPU type you'll deploy to.
Sources. Hugging Face, Hugging Face loves TorchTPU: TPUs, Natively (20 October 2026); torch_tpu repository, GitHub; torch-tpu on PyPI; Google Developers Blog, TorchTPU: Running PyTorch Natively on TPUs at Google Scale.
