Google TorchTPU: PyTorch Finally Runs Natively on Google's AI Chips

Google announced TorchTPU yesterday, and for PyTorch developers this is potentially huge. Finally, native PyTorch execution on Google's Tensor Processing Units — no JAX translation layer, no awkward workarounds, just standard PyTorch code running on Google's custom AI silicon.

What it does:

TorchTPU is a backend for PyTorch XLA that compiles PyTorch operations directly to TPU-optimized kernels. It supports distributed training, mixed precision, and gradient checkpointing. Google claims near-parity with JAX performance on TPUs while maintaining full PyTorch compatibility.

The catch:

It's currently limited to Cloud TPU v5e and v6e instances. Older hardware isn't supported. And while the API looks like standard PyTorch, you'll still need to understand TPU topology and memory layout to get good performance. The 'it just works' promise only applies if you already know what you're doing.

Benchmarks (Google's numbers, grain of salt):

• ResNet-50 training: 1.15x faster than JAX equivalent
• LLaMA-2 7B fine-tuning: comparable throughput, 8% lower memory usage
• BERT pre-training: 0.95x JAX speed (slightly slower)

So What?

TorchTPU matters because it removes the biggest friction point for adopting Google Cloud TPUs: JAX. Most ML engineers know PyTorch. Very few know JAX. If TorchTPU delivers on its promise, Google just made their cloud a lot more attractive to the PyTorch majority.

But it's early days. The real test will be whether complex models (diffusion, multimodal) work without hand-tuning. For now, it's promising but not a no-brainer.