Text Crystallizes from Noise: I Tried Running the Diffusion LLM 'DiffusionGemma'
Standard LLMs write text sequentially from left to right.
They predict the next word, then the next, moving from front to back, like saying, 'Once upon a time...'.
However, DiffusionGemma is a little different.
At first, the entire canvas of text appears as a jumbled mess. From there, it repeatedly removes noise to bring it closer to a coherent textual form.
It is similar to how image generation AI takes a blurry image and makes it gradually clearer. It is doing that with text.
When I actually tried running it, it looked quite mysterious.
First, I want you to look at this.
This is during the generation process when I asked it to 'write a short story about a cat traveling through space'.
At first, what comes out is hard to call text.
`Silver silver,,,, of of of of of of of of of of...`
It has no meaning yet. However, as the steps progress, words start to emerge here and there.
`Silver hull`, `black cat`, `Luna`.
As it progresses further, the skeleton of the sentence is determined.
`Inside the silver hull, a black cat named 'Luna' was sitting.`
And finally, it takes a form that can be read as a story.
The stars, scattered like jewels, shone like particles of light tempting their prey.
It doesn't feel like it's writing one character at a time from the left.
First, the outline of the entire text exists, and it gradually comes into focus. It is truly a movement where 'text crystallizes from noise'.
What is DiffusionGemma?
DiffusionGemma is a block diffusion language model from Google DeepMind.
I tried the GGUF version of `diffusiongemma-26B-A4B-it`. Although it has 26B total parameters, it is an MoE model, so only a portion of it is active at any given time.
Autoregressive models, commonly used in standard chat AIs, predict the next token one by one.
In contrast, DiffusionGemma uses fixed-size blocks and generates text by repeatedly updating the entire block.
Roughly speaking, the differences are as follows:
| Type | Generation Method |
|---|---|
| Standard LLM | Outputs the next token one by one from left to right |
| Diffusion LLM | Refines the entire text block repeatedly until completion |
This difference leads to a visual effect like a GIF, where the whole text gradually takes shape.
It is still a bit difficult to run
This is important: as of June 14, 2026, in my environment, DiffusionGemma did not work with standard Ollama or Lemonade.
The reason is that the general `llama-server` does not yet support this diffusion architecture.
This time, I built the DiffusionGemma-compatible branch of `llama.cpp` myself and used the dedicated `llama-diffusion-gemma-visual-server`.
The environment is as follows:
| Item | Details |
|---|---|
| GPU | Radeon RX 9060 XT 16GB |
| OS | Windows 11 |
| Backend | Vulkan |
| Runtime | llama.cpp DiffusionGemma-compatible branch |
| Model | Q4_K_M / IQ4_XS |
It is not yet at the stage where it can be easily accessed via an OpenAI-compatible API. I wrapped the proprietary stdin/stdout protocol in Python to keep the model resident in memory.
What I found interesting
The most interesting part is being able to see the generation in progress.
With standard LLMs, you can see the text gradually extending through streaming display.
But that is merely 'writing from left to right'.
In the case of DiffusionGemma, the entire text exists in an incomplete state and is rewritten multiple times.
When humans write, we don't necessarily spit out a perfect sentence from left to right from the start. We often have a vague overall picture in our minds and refine it into shape as we go.
The generation process of DiffusionGemma looked a bit similar to that feeling.
Of course, this does not mean that the internal workings of the model are the same as a human's. However, the way the output appears is quite different from autoregressive models.
How was the speed?
It worked even on a 16GB AMD GPU.
However, it is not fast if you run it without thinking. How you use VRAM was quite important.
With the initial stable settings, it was about 1100ms per step. From there, by using a smaller quantized model and increasing the number of layers loaded onto the GPU, I was finally able to shorten it to about 632ms/step.
This tuning part will be long, so I will separate it into another article.
To write just the conclusion, with diffusion LLMs, 'high GPU usage' does not necessarily mean 'fast'. If it overflows from VRAM, the GPU will look busy but will actually be slower.
Summary
DiffusionGemma's generation looks quite different from typical LLMs.
Instead of text extending from the left, the entire text, which looks like noise, gradually takes on a meaningful shape.
Just watching that intermediate process was a quite interesting model.
The execution environment is not yet refined. It is not at the stage where it runs normally with Ollama or Lemonade; it requires self-building or a dedicated client.
Even so, it was a harvest that a 26B class diffusion LLM ran on a 16GB consumer AMD GPU.
In the next article, I will write about the tuning results of how fast I was able to make it in a Radeon 16GB environment, packing VRAM and NGL.
Recommended tags: #LocalLLM #DiffusionGemma #GenerativeAI #AMD #Radeon