Overview of Kimi K2

I found the following article interesting, so I have summarized it briefly.

1. Kimi K2

“Kimi K2” is the latest “Mixture-of-Experts” model, featuring 32B active parameters and 1T total parameters. Among non-reasoning models, it achieves state-of-the-art performance in cutting-edge knowledge, mathematics, and coding. Furthermore, because it is meticulously optimized for agentic tasks, “Kimi K2” does not just provide answers; it takes action.

Today, we are open-sourcing the following models:

2. Benchmarks

2-1. Kimi-K2-Base

The table below summarizes the performance of “Kimi-K2-Base.” Across a wide range of tasks, it demonstrates performance equal to or better than state-of-the-art open-source pre-trained models. “Kimi-K2-Base” shows particularly excellent performance in knowledge-intensive and reasoning benchmarks, and exhibits superior performance in natural language understanding, mathematics, and code generation.

2-2. Kimi-K2-Instruct

The table below summarizes the performance of “Kimi-K2-Instruct.” It shows that the model performs equal to or better than the latest open-source and proprietary models across diverse tasks. This model demonstrates excellent performance in knowledge-intensive and reasoning benchmarks, and achieves outstanding results in natural language understanding, mathematics/science, code generation, and the use of agent tools.

3. Open Agentic Intelligence

Pre-training is a crucial foundation for “Agentic Intelligence,” establishing the priors that make exploration via Reinforcement Learning (RL) manageable, efficient, and generalizable. However, as Ilya Sutskever has pointed out, human data is a finite “fossil fuel,” and its growth lags significantly behind the pace of compute. Therefore, token efficiency during pre-training becomes a new, critical coefficient in AI scaling laws.

Post-training is critical in the “Era of Experience” (David Silver, Richard Sutton, 2025). In this era, LLMs increasingly learn from the interactions they generate themselves, and by gaining rewards, they can be freed from the limitations of human data and potentially surpass human capabilities.

“Kimi K2” was born precisely from these insights.

4. MuonClip Optimizer

Without considering rigor, given a nearly finite pre-training dataset and a fixed model configuration, a more token-efficient optimizer generates more intelligence. In MoonshotAI’s previous research, “Moonlight,” it was demonstrated that the “Muon” optimizer significantly outperforms the AdamW optimizer widely used in LLM training.

“Kimi K2” was designed to further scale up “Moonlight,” which adopts an architecture similar to “DeepSeek-V3.” Based on scaling law analysis, we reduced the number of heads for long-context efficiency and increased the sparsity of the MoE for better token efficiency. During the scaling-up process, we faced a deep-seated challenge: training instability caused by exploding attention logits. In MoonshotAI’s experiments, this occurred more frequently with “Muon” and less frequently with AdamW. Existing solutions such as logit soft-capping or query-key normalization proved insufficient.

To solve this problem, we introduce the “MuonClip Optimizer,” which improves “Muon” using the proposed qk-clip technique. Specifically, qk-clip stabilizes training by directly rescaling the query and key projection weight matrices after the “Muon” update, controlling the scale of attention logits at the source.

Experiments showed that “MuonClip” can effectively prevent logit explosions while maintaining downstream task performance. In fact, “Kimi K2” was pre-trained on 15.5T tokens using “MuonClip” with zero training spikes. This demonstrated that “MuonClip” is a robust solution for stable, large-scale LLM training.

5. Agent Capabilities

The enhanced agent capabilities of "Kimi K2" stem from two key aspects: large-scale agent data synthesis and general-purpose reinforcement learning.

5-1. Large-Scale Agent Data Synthesis for Tool-Use Learning

To train the model in advanced tool-use capabilities, we developed a comprehensive pipeline inspired by ACEBench. This pipeline simulates real-world tool-use scenarios at scale. Our approach systematically evolves hundreds of domains containing thousands of tools (including both actual MCP (Model Context Protocol) tools and synthetic tools), generating hundreds of agents equipped with diverse toolsets.

All tasks are rubric-based, allowing for consistent evaluation. Agents interact with simulation environments and user agents to create realistic multi-turn tool-use scenarios. An LLM judge evaluates the simulation results against the task rubric, filtering for high-quality training data. This scalable pipeline generates diverse, high-quality data, paving the way for large-scale rejection sampling and reinforcement learning.

5-2. General-Purpose Reinforcement Learning

A key challenge is applying reinforcement learning (RL) to tasks with both verifiable and unverifiable rewards. Typical examples of verifiable tasks include mathematics and competitive coding, whereas writing research reports is generally considered unverifiable. Beyond verifiable rewards, our general-purpose reinforcement learning system employs a self-judgment mechanism where the model acts as its own critic, providing scalable, rubric-based feedback for unverifiable tasks.

Meanwhile, on-policy rollouts with verifiable rewards are used to continuously update the critic, which continues to improve its evaluation accuracy against the latest policy. This can be viewed as a method of using verifiable rewards to improve the estimation of unverifiable rewards.

6. Getting Started

6-1. Try Kimi K2 at kimi.com

Starting today, Kimi web and mobile users can select and use the new "Kimi K2" for free. Currently, MCP features for web and apps are still under development. We plan to begin rolling them out within the next few weeks. In the meantime, you can try out agent features in Researcher. Vision capabilities are not yet supported in "Kimi K2".

6-2. Use Kimi K2 via API

The Kimi platform provides an OpenAI/Anthropic-compatible interface, making it easy to adapt existing applications to Kimi K2. For more information on tool calling for building agent applications, please refer to platform.moonshot.ai

6-3. Self-host Kimi K2

We recommend running "Kimi K2" with one of the following inference engines: "vLLM," "SGLang," "KTransformers," or "TensorRT-LLM." For detailed deployment instructions, please refer to the GitHub repository

6-4. Future Outlook

"Kimi K2" serves as a solid foundation for open agent intelligence, but general-purpose agents utilize more advanced features such as reasoning and visual understanding. We plan to add these features to "Kimi K2" in the future.

6-5. Limitations

When handling tasks that are difficult for reasoning or have unclear tool definitions, the model may generate excessive tokens, leading to truncated output or incomplete tool calls. Furthermore, enabling tool use may result in performance degradation on certain tasks. When building complete software projects, one-shot prompting may perform worse compared to using K2 within an agent framework.