I tried the Qwen3.6-27B-MTP-GGUF released by unsloth: 'Generation is fast' - Multi-Token Prediction

Impressions

In the example I tried this time, it was about 1.6 times faster.
It's quite comfortable. From the feel of trying it several times, I don't see any obvious performance degradation.
This is good.

However, the model is listed as "unsloth/Qwen3.6-27B-MTP-GGUF", but in the llama-server command it is "unsloth/Qwen3.6-27B-GGUF-MTP", so the position of MTP is different. Why?

Model

Build a dedicated llama.cpp

As per the model card.

git clone -b mtp-clean https://github.com/am17an/llama.cpp.git

cmake llama.cpp -B llama.cpp/build -DBUILD_SHARED_LIBS=OFF -DGGML_CUDA=ON

cmake --build llama.cpp/build --config Release -j --clean-first --target llama-cli llama-server

cp llama.cpp/build/bin/llama-* llama.cpp

Execution

Version without image input

I'm trying this one this time.

./llama.cpp/llama-server \
  -hf unsloth/Qwen3.6-27B-GGUF-MTP:UD-IQ3_XXS \
  -ngl 99 -c 32768 -fa on -np 1 \
  --spec-type mtp --spec-draft-n-max 3 \
  -ctk q8_0 -ctv q8_0 --no-mmproj

VRAM 14.88 GB

Version with image input

./llama.cpp/llama-server \
  -hf unsloth/Qwen3.6-27B-GGUF-MTP:UD-IQ3_XXS \
  -ngl 99 -c 32768 -fa on -np 1 \
  --spec-type mtp --spec-draft-n-max 3 \
  -ctk q4_0 -ctv q4_0

VRAM 15.46 GB

Generation speed

82.57 tokens per second

prompt eval time =     114.66 ms /    18 tokens (    6.37 ms per token,   156.99 tokens per second)
       eval time =  150643.49 ms / 12438 tokens (   12.11 ms per token,    82.57 tokens per second)
      total time =  150758.15 ms / 12456 tokens

It's somehow pleasing that it gets faster as generation progresses.

In terms of speculative decoding, n-grams also increased in speed when conditions were met, but those conditions were quite limited.
However, MTP seems to be working well.

Comparison

Official llama.cpp.

./build/bin/llama-server \
  -hf unsloth/Qwen3.6-27B-GGUF:UD-IQ3_XXS \
  -ngl 99 -c 32768 -fa on -np 1 \
  -ctk q8_0 -ctv q8_0 --no-mmproj

Results for the same prompt.

51.22 tokens per second

prompt eval time =     115.97 ms /    18 tokens (    6.44 ms per token,   155.21 tokens per second)
       eval time =  223138.02 ms / 11429 tokens (   19.52 ms per token,    51.22 tokens per second)
      total time =  223253.98 ms / 11447 tokens