Google DeepMind has added a new model to the open-source Gemma 4 family: DiffusionGemma. According to the company’s own tests, the model reaches about 700 tokens per second on an Nvidia RTX 5090 and more than 1,000 tokens per second on a single Nvidia H100, roughly 4 times the speed of an autoregressive Gemma model of similar size. The trade-off is clear in Google’s published results: DiffusionGemma scores below standard Gemma 4 across the public benchmarks that have been released so far.
A text model built on diffusion instead of left-to-right decoding
Most language models generate text autoregressively, producing one token at a time from left to right. DiffusionGemma uses a different method. It begins by filling the output space with placeholder tokens, then refines the full sequence through repeated denoising passes until the text converges into a final result. The process is closer to how Stable Diffusion builds images than how GPT-style models write text.
Google says this architecture brings measurable speed gains on local hardware. The model is being released under the Apache 2.0 license, making it available for developers and researchers to use, fine-tune, and study.
26B total parameters, but only 3.8B active during inference
DiffusionGemma uses a mixture-of-experts, or MoE, design. The model has 26 billion total parameters, but only 3.8 billion parameters are activated for a given inference run. That lowers the active compute burden, and Google says the model can run within 18GB of VRAM on high-end GPUs, especially after quantization.
Its generation process is also highly parallel. A standard autoregressive model has to wait for one token before computing the next. DiffusionGemma updates the whole sequence across multiple denoising steps and can handle up to 256 tokens in parallel at a time. Google points to tasks like Sudoku as an example, where one correct choice depends on other positions that may not be settled yet. A model that can keep revising the full token set may have an advantage in that kind of dependency-heavy problem.
Why the speedup appears on modern GPUs
Google’s explanation centers on hardware bottlenecks. Autoregressive inference is constrained more by memory bandwidth because model weights need to be read repeatedly for each token. Diffusion-style generation shifts the pressure toward compute, processing many tokens together while reducing the number of memory reads per token on average.
That matters on current GPUs, where raw compute often exceeds available memory bandwidth. In Google’s framing, autoregressive decoding can leave expensive compute units waiting on memory transfers, while diffusion generation keeps more of the hardware busy through larger parallel workloads. The company’s reported numbers are around 700 tokens per second on RTX 5090 and over 1,000 tokens per second on H100. Google also notes that these figures come from internal testing rather than third-party verification, so actual gains may vary by workload and output length.
Lower benchmark quality is the price of higher throughput
The quality side is less favorable. In every public benchmark disclosed so far, DiffusionGemma trails standard Gemma 4. The model’s speed advantage is not coming for free; Google’s own data points to a broad drop in output quality.
That makes the model a specific kind of tool rather than a blanket replacement. For workloads that care more about throughput, local inference on constrained devices, or low latency, DiffusionGemma may be useful. Google also lists inline editing, molecular sequence generation, and mathematical plotting among the scenarios it sees as relevant. For tasks where answer quality matters more, standard Gemma 4 remains the safer option based on the published results.