How Text Generation Becomes '4x Faster'—Google's 'DiffusionGemma' Technology and Usage

Have you ever wondered why LLMs don't generate all their text at once when they create it?

Actually, this is due to the fundamental architecture of LLMs.
Text is always generated one token at a time, referencing the previous tokens—this is how current mainstream models work.

Released by Google on June 10, 2026, 'DiffusionGemma' is an open-weights model designed to overturn that premise.
It adopts a technology called text diffusion, achieving generation speeds up to four times faster than conventional LLMs.

In this article, we will explain the mechanism of text diffusion and the features and usage of DiffusionGemma.

📖 Why do conventional LLMs generate 'one token at a time'?

Current LLMs like ChatGPT and Claude operate on an architecture called 'autoregression'.

In this architecture, all preceding tokens are referenced to predict the next token.
It is a serial process: generate token A, reference A to generate B, then reference A and B to generate C, and so on.

Because the processing is serial, it takes longer as the number of tokens to be generated increases.
This is the fundamental reason why text doesn't appear all at once.

🎨 How is DiffusionGemma 4x faster?

DiffusionGemma applies the concepts of image generation AI (diffusion models) to text generation.

According to Google, it employs a technology called 'Uniform State Diffusion'.
It opens a 'canvas' of 256 tokens filled with random placeholders and refines the tokens in all slots in parallel over multiple steps.

It is closer to the image of drawing the overall outline first and then filling in the details little by little, rather than writing one token at a time in order.
Since it can process 15 to 20 tokens in a single forward pass, the generation speed is significantly improved.

For those familiar with how image generation AI gradually draws a picture from noise, it may be easier to understand if you think of it as an application of that to text.

⚡ Actual generation speed

According to Google's announcement, DiffusionGemma achieves the following speeds:

• H100 (FP8, low batch size setting): 1,100 tokens/second or more

• GeForce RTX 5090: 700 tokens/second or more

This is said to be up to 4 times faster than conventional LLMs.

An NVFP4 version optimized by NVIDIA for RTX is also provided, allowing it to run on high-end consumer GPUs.
Another feature is its high memory efficiency, which allows it to run even on high-performance consumer GPUs.

📦 Specs and basic information

These are the main specifications of DiffusionGemma.

• Base Model: Gemma 4 (26B-A4B MoE architecture)

• Active Parameters during Inference: 3.8B (Total parameters are 26B)

• License: Apache 2.0 (Commercial use permitted)

• Supported Languages: 35+ languages (intended for multilingual NLP tasks)

• Input Format: Multimodal support for text, images, and video

The fact that it is a '26B model but only 3.8B during inference' is due to the MoE (Mixture of Experts) architecture.
Designed so that only a portion of the total parameters are actually used for each inference, it achieves both speed and memory efficiency.

🗣️ What it can be used for

The use cases supported by DiffusionGemma are as follows.

• Conversational AI/Chatbots

• Text summarization

• Code generation and step-by-step reasoning

• Image and document understanding (OCR, charts, PDFs, screen analysis)

• Video content analysis

• Agent workflows (native function calling support)

• Multilingual NLP tasks

It covers almost all use cases handled by standard LLMs, and is expected to be particularly useful in 'situations where speed is a bottleneck'.
It is well-suited for systems where speed is critical, such as real-time chatbots, batch processing for large-scale document summarization, and internal agent reasoning.

🚀 Download and use from HuggingFace

DiffusionGemma is published on HuggingFace and can be downloaded without any special application (a HuggingFace account is required).

The available variants are as follows.

• Standard Version: google/diffusiongemma-26B-A4B-it

• GGUF Version (Quantized): unsloth/diffusiongemma-26B-A4B-it-GGUF

• NVFP4 Version (NVIDIA Optimized): nvidia/diffusiongemma-26B-A4B-it-NVFP4

Inference code and setup instructions are available in the official documentation (ai.google.dev/gemma/docs/diffusiongemma).

Supported frameworks are as follows.

• vLLM (High-speed inference server)

• HuggingFace Transformers

• MLX (Runs on Apple Silicon)

• Unsloth (Specialized for quantization)

• NVIDIA NeMo (For enterprise)

It has been compatible with these frameworks since the first day of release, and is designed to be easily integrated into existing pipelines.

🌏 Usage in Japanese Environments

DiffusionGemma is stated to support multilingual NLP tasks in over 35 languages, and Japanese is included in that scope.

Since it is released globally, it can be used without issues from Japan via HuggingFace.
For Japanese developers and researchers developing systems where text generation speed is a bottleneck, it is one option worth trying.

⚠️ Points to Note at This Time

DiffusionGemma is released as an 'experimental model'.
Integration into commercial production environments should only be done after thoroughly evaluating its quality and stability.

Because its architecture differs significantly from traditional Transformer models, standard inference code for LLMs cannot be used as-is.
You must use the latest versions of supported frameworks (vLLM, HuggingFace Transformers, etc.).

The '4x faster' figure is a measurement taken on an H100 with low batch size settings.
Speeds on typical GPUs or CPUs may differ from this figure.

Also, compared to MiMo Code, which recorded scores surpassing Claude Code on coding benchmarks like SWE-Bench Pro, DiffusionGemma is a model focused on 'speeding up general-purpose LLMs'.
It is appropriate to view it as a 'fast text generation infrastructure' rather than using it as a coding agent.

📝 Summary

DiffusionGemma is an experimental model that changes the fundamental mechanism of text generation.
Instead of the traditional 'serial generation of one token at a time,' it uses an approach of 'refining 256 slots in parallel,' achieving up to 1,100 tokens per second on an H100.

It is available for free download from HuggingFace under the Apache 2.0 license and is compatible with vLLM, MLX, Unsloth, and NVIDIA NeMo.
It supports over 35 languages, and Japanese NLP tasks are also said to be included.

Although it is an experimental model, it is a work that demonstrates the potential for scenarios where high-speed text generation is required.
If you are interested, please download it from HuggingFace and give it a try.

I hope this is at least a little helpful.
Thank you for reading until the end.

#AI #GenerativeAI #AITools #Technology #Google #DiffusionGemma #Explanation #Engineer