[Favorites] Recommended Local LLM Rankings for the Author ① [Late 2026 Edition]
Introduction
This is a series of articles where I periodically introduce the local LLMs I am actually using.
The previous edition (early 2026) can be found here. It also introduces basic selection methods such as quantization.
However, I will stop using a ranking format for late 2026.
In the parameter scale of LLMs that run on gaming PCs, the two giants, Google Gemma4 and Alibaba Qwen3.6, have become so prominent that minor models are no longer putting up much of a fight. (In reality, there are still a few here and there, but they seem unable to surpass the performance in the same scale range.)
On the other hand, I believe the current trend in the community is custom models based on Gemma4/Qwen3.6, which utilize improved performance of these models or training data created with smarter models.
Therefore, I would like to introduce the Gemma4 and Qwen3.6 series models that I am actually using as my primary tools.
Gemma4 Series
Overseas, it is not as popular as Qwen. The reason is that its agent performance (such as tool calling) and coding performance are slightly inferior.
However, in Japan, it has overwhelming Japanese language performance, so for non-IT professionals (Japanese users), I think Gemma4 is the best choice.
Gemma4 includes the advanced quantization released by Google officially, the QAT version, and the MTP version, which achieves 2-3 times or more speedup without sacrificing performance. Of course, QAT+MTP can be used simultaneously.
* However, at present, you cannot easily use the Gemma4 MTP version with LM Studio or ollama. You need to run it directly using Unsloth Studio, llama-server (llama.cpp), etc.
Gemma4 31B
It is the highest-performing LLM currently running on gaming PCs, but it requires a large amount of VRAM. Even in my RTX5060ti 16GB x 2 = 32GB environment, it cannot be operated at 256k tokens. 128k was the limit with QAT+KV cache Q8_0.
I am using the unsloth version of qat-31B.
* However, since the "ollama pull" command results in an error, it was safer to create an ollama model file after downloading it individually.
While it is about 20-30 token/s with standard ollama on an RTX5060ti 16GB x 2 + Linux environment, by using a
forceful method, I have enabled QAT+MTP+split-tensor and am operating it at 70-90 token/s.
Gemma4 26B MoE A4B
This is the most suitable model for many gaming PC users. Although it has a large parameter count of 26B, it only uses 4B selectively, which provides a significant advantage in calculation speed.
Because it runs stably in many environments, it is the model I use primarily.
Even with a hybrid CPU+GPU or CPU-only setup, you can achieve practical speeds (10–50 token/s) for chat purposes. If you have a powerful CPU, you can get 30 token/s with just the CPU.
Generally, you will be using quantized models like Q3–Q4_K_M, or QAT versions.
However, while there is a possibility of issues or bugs in llama.cpp (or GGUF conversion), there are many reports that even with the same 4-bit quantization, the performance of Q4_K_M is better. I feel the same way.
QAT version < Q4_K_M
※ Unlike the 31B, the 26B is an ultra-lightweight model running on 4B parameters, so (on mid-range GPUs) there is no significant difference in parallel processing or MTP. In fact, it might even be slower. In reality, it becomes slower with the llama-server included in the current version of Ollama.
Example: RTX 3060 12GB x2 (Windows 11, no MTP, no split tensor)
Example: RTX 5060ti 16GB x2 (Linux, no MTP, no split tensor)
If you adjust the CPU offloading correctly, you can achieve a stable 50 token/s even with a single RTX 5060ti 16GB VRAM card for around 128k context.
However, since offload adjustment is difficult in Ollama, I recommend running it in LM Studio.
※ If you use the "forceful method" used for the 31B mentioned above, you can directly control the llama-server arguments, so it is also possible with Ollama + OpenWebUI.
Gemma4 12B
On a mid-range gaming PC, another option is the 12B. Since it is not an MoE, it runs slower than the 26B, but because the total amount of model weight data (static data) is small, it runs at almost full specs with 12GB–16GB of VRAM.
Even with 8GB of VRAM, it will fit on the GPU if you adjust the quantization size and context length.
However, even if it is comfortable, it is a dense model, so it is slow (heavy). If everything fits in VRAM, I think the 26B A4B is overwhelmingly faster.
Also, while I think the 26B has higher overall capability, the 12B can be smarter in certain fields.
RTX5060ti 16GB x2
RTX3060 12GB x2
In my (Ollama) environment, I cannot expect speedups from parallelism or MTP, so it is not very convenient. A single GPU might be faster.
Since the model size is small, I think it is an option for environments where the 26B does not run because system memory (not VRAM) is absolutely insufficient.
* Although I haven't verified it, while 12B performance on my mid-range GPU is lackluster, I believe that even with small VRAM of 8-12GB, if you have a high-end GPU with powerful VRAM bandwidth, there is a high probability of achieving better results. In my environment, GPU power utilization is clearly poor, indicating a major bottleneck. It is a state where there is available CUDA for computation, but data transfer cannot keep up.
Gemma4 E4B
From this class onwards, it becomes possible to run it somehow even on entry-level gaming PCs and laptops.
Even with the same level of quantization, the file size might be larger than the Gemma4 12B version, but since it contains data that does not need to be loaded into VRAM, it runs with less VRAM memory in supported operating environments like the latest ollama or LM Studio.
E4B or models with around 8B parameters are, I think, the smallest practical models that also have versatility. Gemma4 has an even smaller model called E2B, but while it can be used for simple conversation purposes, I think logical judgment is a bit difficult.
* Although it says E4B, it is not 4B; in reality (file size), it is in the 8B class, so when comparing with others, around 8B (Qwen3.5 9B for Qwen) is its rival.
Even with University of Tokyo entrance exam questions, it reaches the correct answer if reasoning is included.
However, it does not reach the correct answer. (Without reasoning)
Qwen3.6 series
Qwen's weakness is clear: it (sometimes) fails at Japanese generation. It is especially stressful because it is hard to tell when it outputs Simplified Chinese. If it is for summarization or editing work in Japanese, it is no match for Gemma4 at all.
However, because of its tool-calling performance, coding performance, and ease of tuning, many custom models for AI agents and coding are created daily in communities where Japanese is irrelevant.
I also rarely use the base model and instead use custom models.
In order of intelligence
In the case of Gemma4, the parameter scale and intelligence are proportional, so it is easy to understand, but Qwen is a little different. Arranged in order of intelligence (high performance), it is as follows. Note that 27B (dense) and 35B (MoE) are reversed.
Qwen3.6 27B > Qwen3.6 35B > Qwen3.5 9B > Qwen3.5 4B
Qwopus3.6-27B-Fusion
This is a Qwen3.6-27B-based model. However, since 31B is the rival dense model for Gemma4, it cannot be used comfortably without 24GB to 32GB of VRAM.
Qwopus is not a new model, but one of the standard popular models. As the name suggests, it is mainly trained on Claude Opus's reasoning.
Recently, a model (Fusion) that merges the regular version and the code version was released, and I think it has quite good stability.
* Unlike Gemma4, Qwen3.6 can use MTP in LM Studio and Ollama. However, it only provides about a 1.5x speedup.
Modified ollama using the same method as Gemma4-31B (MTP+split-tensor)
Ornith-1.0 35B
Qwen3.6 35B is an A3B-based model. Since it is an MoE, it is comparable to the Gemma4-26B. Like Qwopus, it is a standard, popular model. There is also a 9B version.
Because it is an A3B MoE, if you have a powerful CPU and enough system memory, it can perform well even when running on the CPU.
Like Qwopus, it is a model intended for AI agents and coding.
Gemma4 26B cannot handle MTP well (in llama.cpp), but with Qwen3.6 35B, you get about a 1.5x effect. However, it is not very effective in hybrid mode.
Macaron-V1-Tall 35B
This is a recently released model, but expectations are high.
Summary
Recently, the release of massive models like GLM5.2 and K3, which are open but physically impossible to run on home (gaming) PCs, has been a hot topic, but this is not completely unrelated.
The more free, high-performance models are released openly, the more they are used for research and training, which raises the performance floor for smaller models.
For example, while Qwopus certainly uses the results from Opus, it cannot be used as-is. Reasoning from commercial services is encrypted and obfuscated (to prevent guardrail analysis and misuse), so it cannot be used efficiently for training as-is, which leads to performance degradation.
On the other hand, open models are output as-is, which provides a significant advantage for community training and the creation of new models.
If open models have reached the point where they rival closed frontier models, I believe the emergence of even higher-performance small and medium models will accelerate in the future.
Below is a [PR] for related articles.