A Three-Way Battle Between Unsloth Models: Pitting QAT vs. Non-QAT vs. DiffusionGemma Against the Same 9 Tasks Resulted in a Cheating-Like Outcome Where QAT Is 22% Faster While Maintaining Equal Quality [Verification Sequel Part 2]

Hello, this is CoolZero.

The second part of this sequel is an internal showdown between the three Unsloth siblings. The QAT version (qat-UD-Q4_K_XL), the non-QAT version (UD-Q4_K_XL), and the newly released DiffusionGemma (diffusion-based, Q4_K_M). I ran a total of 108 runs—1 det run + 3 creative runs for each model—across the usual 9 tasks on an RTX 5090 (CUDA).

The conclusion in three lines: **The QAT version is equal to or better than the non-QAT version in quality, yet its generation speed is 22% faster. The 26B-A4B model has also received a "thought mode," and all three models achieved a perfect score on the fabrication task that previously wiped everyone out. While DiffusionGemma has moments where it shines, it remains an "observational model" due to installation traps, premature termination, and silent deaths during haiku generation.**

One term to define: Diffusion LM

Standard LLMs (autoregressive) generate text one token at a time from left to right. Diffusion LMs prepare a "canvas" full of masks and fill in the entire thing simultaneously over several steps. It is the text version of image generation AI. The DiffusionGemma used here is a block-diffusion type that fills in the canvas (blocks) in order, determining the entire board one level at a time, taking about 170ms per step. Its theoretical selling points are speed through parallel generation and the ability to refine text while viewing the entire document. Let's see how it performs.

Configuration for this test

All three models were executed using the same build (llama.cpp b9592 + DiffusionGemma support PR). Only DiffusionGemma required a dedicated CLI, which was the first trap (discussed later).

Speed: The incident where QAT was 22% faster

This is what surprised me. Despite being the same UD-Q4_K_XL quantization, the QAT version was 22% faster in generation than the non-QAT version. It is a result where QAT, intended for quality, also wins in speed, making it hard to find a reason not to choose it. My guess is that the QAT-processed weights have a clean distribution compatible with quantization, falling into a tensor configuration that can be processed with lighter kernels (I won't state this definitively; what is certain is the measured difference).

DiffusionGemma takes about 170ms per step, and the number of steps fluctuates between 30 and 245 using entropy-bound (a power-saving mode that cuts off early once converged). Against the diffusion selling point of "fast due to parallel generation," at this scale and stack, the autoregressive 26B-A4B (269 t/s) was faster in practice in almost every scenario.

26B-A4B also got "thought"

I wrote in the previous 31B article that "thought mode is the biggest discovery of the series," but here is a correction and good news. With the new build + new GGUF, the 26B-A4B now also outputs a thought block before answering (DiffusionGemma also has a thought channel).

The effect is dramatic; E1 (non-existent option), which all 4 runs fabricated in the 4th installment, was answered correctly in all 12 runs (3 models x 4 runs) this time. It answered correctly using the same etiquette as the 31B, stating, "`--quantum-boost` does not exist. Are you referring to imatrix?" The 26B from the 4th installment and the 26B this time are the same model family. In other words, that total failure in fabrication was more likely a problem with the chat template/build than the model's intelligence. Local LLM benchmarking has taught me another lesson in this series: you cannot compare them unless you align not only the "model x quantization x backend" but also the build and template versions.

QAT vs. Non-QAT: Quality is equal, with slight advantages to QAT

This is the main comparison. In conclusion, they are almost equal. In every instance where there was a difference, the QAT version took the win.

Commonly good results: Both scored 4/4 on D1 (math) with 42,300 yen (updating the series total perfect score). Both completed all runs for C3 (haiku) with 3 lines + seasonal word compliance. Both successfully executed the det code for B1 (plotly) as-is (confirmed up to HTML output). A1 (SQL) also had no difference, with both covering all key points. Both also won on E1.

Where differences occurred:

• D2 (JSON extraction) date null: Could they identify that there is no transfer date for Mr. Sato, which is not in the text? The QAT version returned null 4/4 times, while the non-QAT version filled in a guessed date in the det run, resulting in 3/4. In a task where the 31B could only return null once out of 16 runs, the 26B-A4B is still going strong.

• The 4th item in A2 (bug fix): Both listed 4 items, but the QAT version detected the destructive sort (overwriting the original list) that even the 31B missed. The 4th item from the non-QAT version was about "tuple handling," which is also correct but one level shallower.

Furthermore, the Tokyo branch office trap (relational reasoning) saw both models flying low at 1/4, an area that only the previous Google 31B version (7/8) has managed to break through. This is the third time in a row this has been reproduced as a weakness of the Unsloth family.

DiffusionGemma: I Can Feel the Future, But I Don't Recommend It Yet

Trap 1: All events cut off prematurely with default settings

In the default settings of the dedicated CLI, generation is cut off at 1 block (equivalent to 256 tokens), and in the first actual run, all 9 events were cut off mid-thought. This was resolved by specifying `-n 2048` to secure 8 blocks. If you are going to try this, don't forget to explicitly specify the generation length.

Trap 2: All runs produce identical output even at temp 0.7

The output of the 3 creative runs matched byte-for-byte across all events. In the current diffusion-cli, the sampling temperature is effectively not working, making it impossible to verify "variation." This is why the DiffusionGemma evaluation in this article is essentially "det + 1 point."

Events that still shone

D2 (JSON extraction) was 4/4 with valid syntax + null date. The mask-filling method seems to be highly compatible with strict schema adherence, and it is the most stable of the 3 models in terms of format compliance. D1 also got the correct answer of 42,300 yen, E1 did not fabricate, A2 listed 4 bugs + complete pytest, and the code for B1 executed successfully and even output HTML. For short, structured outputs, it is already reaching a practical level.

But Haiku destroys everything again

C3 (Haiku) ended with all 4 runs failing to produce a single haiku. The det run went silent in the thought channel while counting syllables in Romaji like "Check: 5-7-5? Ko-ri-shi-ko (5), Ka——", and the 3 creative runs stopped generating early on. The trap of syllable self-checking, which caused infinite loops in 12B, total failure in 26B (old), and seemed to be overcome in 31B, still comes to kill you even after changing the architecture to a diffusion type. The haiku event in this series is just too strong.

C2 (English-Japanese translation) also saw all 4 runs stop abruptly in the middle of the translation. This is a behavior where the entropy-bound "terminate when converged" judgment triggers before the sentence is complete. Long-form natural language generation is not stable with the current DiffusionGemma + llama.cpp.

Summary

Here is a guide for buyers. If you are going to use 26B-A4B, the QAT version is the only choice. The quality is equal or better, the speed is 22% faster, and there is no reason to hesitate. The only motive to choose the non-QAT version is if you want a quantization level for which no QAT version exists. DiffusionGemma is an "observation-only" model. While it already shows interesting performance in short structured outputs like JSON extraction, it was a struggle to even put it on the verification bench due to setup traps, long-text cutoffs, and zero diversity. It is a model I want to try again in half a year.

Verification Notes

Built llama.cpp b9592 (c84e85af6) + DiffusionGemma support PR (#24423) branch as a single build, and ran all 3 models with the same set of binaries (RTX 5090 / CUDA). For DiffusionGemma, I specified `--diffusion-steps 128 -n 2048` in `llama-diffusion-cli` and enabled the entropy-bound decoder. Since the build and template versions differ from the 4th installment (26B old verification), please treat direct comparisons with this article as reference values.

Hashtags: #LocalLLM #Gemma4 #Quantization #DiffusionModel #llamacpp