[LM Studio] Does the new 'MTP' feature really make local LLMs faster? Verification and tips for use
LM Studio has been updated to version 0.4.14, and as a highlight feature, MTP Speculative Decoding has been implemented in a stable version.
While it is rumored that this technology can "nearly double generation speed," there is actually an unexpected pitfall where it can become slower depending on the conditions as well.
In this article, I will explain the mechanism of MTP and the "points that Japanese users should be aware of" that were revealed through actual device verification.
1. What is MTP Speculative Decoding?
"Speculative Decoding" is, in a nutshell,
**AI-driven 'look-ahead calculation'**.
Normally, an LLM calculates one character at a time in sequence, but with this technology, the AI "predicts" the words that follow and creates a draft. The main AI then verifies that draft, and if it is "correct," it adopts it as-is, allowing multiple characters to be output in a single calculation, dramatically improving speed. In particular, MTP (Multi-Token Prediction) is characterized by having this "look-ahead-only head (MTP head)" built into the model itself, and it can be used with compatible models such as Gemma 4, DeepSeek V3, and Qwen 3.6.
2. How to configure it in LM Studio
In LM Studio 0.4.14 and later, you can configure it when loading a model or from "Advanced Configuration" in the sidebar.
MTP Speculative Decoding:
The main switch. Turn it ON if the model is compatible.MTP Max Draft Tokens:
Determines how many steps ahead the AI will jump to create a draft
(default is 2).MTP Min Draft Tokens:
The minimum number of tokens required for verification (default is 0).
Basically, the **default (ON / 2 / 0)** is considered the golden setting with the best balance.
3. [Important] If the 'acceptance rate' is low, it might actually be slower!?
This is the most important point to note. As a result of actual device verification, a phenomenon has been confirmed where "token generation speed is slower when MTP is ON than when it is OFF."
The cause lies in the "Draft Tokens Accepted" rate. If the AI's look-ahead is correct, it becomes faster, but if it is wrong, an overhead occurs where "the main model must verify and discard the incorrect draft and recalculate."
Generally, to improve speed, an acceptance rate of 60% to 70% or higher is required. If the verification result is lower than that, it will be faster to keep it OFF due to the cost of redoing the work.
4. Verification details
The verification used qwen3.5-4b-mtp and qwen3.5-9b-mtp, and checked the tokens/second and acceptance rate with MTP=ON and OFF using the following prompts.
Please write a Python program that "prompts the user to input a number from 1 to 100 and plays a number guessing game."
For beginners, be sure to include detailed Japanese comments for each line of code, and make it a polite explanatory code of 200 lines or more.
Unnecessary greetings or introductions are completely unnecessary. Please output only the code.
Please create 3 patterns of business email templates for reporting recovery and apologizing to customers when a system failure occurs, categorized by situation (initial response, interim report, full recovery).
For each pattern, please output it as a formal and polite long-form business document, including the subject line, body, and future countermeasures.
Please explain the plot of the story of "Momotaro" in great detail, divided into introduction, development, twist, and conclusion.
In particular, describe the daily life of the old man and woman, the scene where the peach comes floating down, the scene where he gathers companions, and the battle on Onigashima, each with a rich, long-form description, and output it so that the whole thing becomes a large volume of text.
5. Verification Results
As a result of the verification,
the tokens/second was slightly higher when OFF
The acceptance rate when ON was about 55%
. It was found that it is difficult to obtain the benefits of MTP with this prompt.
# qwen3.5-4b-mtp
## ケース1
* ON ：161.01 トークン/秒、53.7% draft tokens accepted
* OFF：169.61 トークン/秒
## ケース2
* ON ：165.04 トークン/秒、55.0% draft tokens accepted
* OFF：170.10 トークン/秒
## ケース3
* ON ：146.08 トークン/秒、48.8% draft tokens accepted
* OFF：170.26 トークン/秒
# qwen3.5-9b-mtp
## ケース1
* ON ：120.19 トークン/秒、54.7% draft tokens accepted
* OFF：120.02 トークン/秒
## ケース2
* ON ：118.22 トークン/秒、54.1% draft tokens accepted
* OFF：119.40 トークン/秒
## ケース3
* ON ：120.77 トークン/秒、54.7% draft tokens accepted
* OFF：119.66 トークン/秒
Japanese text generation: Since many current models are trained primarily on English and code, does the prediction accuracy for Japanese tend to decrease?
Highly free (creative) writing: In contexts where there are countless choices for the next word, is the AI more likely to make incorrect predictions?
Summary: MTP is not a 'magic wand'
MTP is a very promising technology, but in the current Japanese environment, it has a strong aspect of being an 'experimental feature'.
First, please check the 'Draft Tokens Accepted' rate for your environment on the LM Studio status screen. If it is below 50%, it seems that the smart way to use it for now is to simply turn it OFF, but since there is not that much of a difference, I will leave it at the default.
いいなと思ったら応援しよう！
よろしければ応援お願いします！