Google Announces DiffusionGemma: Accelerating Text Generation by Up to 4x

3-Minute Summary

Google has released an experimental open model called "DiffusionGemma."
Instead of generating one token at a time, it generates blocks of text in parallel and iteratively refines them.
It is reported to be up to 4x faster on dedicated GPUs.
However, the quality is lower than the standard Gemma 4.
AI selection has shifted from performance rankings to designing for speed, quality, and device constraints.

What Happened

Google DeepMind has released "DiffusionGemma," an experimental open model that adopts a text diffusion approach.

While typical LLMs generate one token at a time from left to right, DiffusionGemma generates 256-token blocks in parallel and refines the entire text iteratively. It is described as being up to 4x faster than standard methods on dedicated GPUs, exceeding 1,000 tokens per second on an H100 and 700 tokens per second on an RTX 5090.

The model is a 26B total Mixture of Experts, activating 3.8B parameters during inference. It runs on approximately 18GB of VRAM when quantized and has been released under the Apache 2.0 license.

However, the output quality is lower than the standard Gemma 4, and Gemma 4 is recommended for use cases where production-grade quality is the priority. The benefits of the speedup are significant for low-to-medium parallel processing on dedicated GPUs, and similar effects may not be achieved on Apple Silicon.

Why It Matters

This is an important experiment that attempts to change the generation method of LLMs themselves to reduce wait times for local AI.

The bidirectional nature, which allows for refinement while viewing the entire text, may be suitable for use cases where context needs to be handled simultaneously, such as inline editing, code completion, mathematical structures, and amino acid sequences.

On the other hand, there is a clear trade-off between speed and quality. When selecting a model, it is necessary to consider not just the "smartest model," but also response speed, device constraints, and the quality required for the specific application.

Study Redesign Notes

If you have a high-performance GPU, you can test local high-speed text editing or code completion.
It can be used for research into internal auxiliary tools that run with low latency without sending sensitive data externally.
You can consider a division of roles, such as using DiffusionGemma for drafting and candidate generation, and Gemma 4 for final quality.

URL

https://blog.google/innovation-and-ai/technology/developers-tools/diffusion-gemma-faster-text-generation/