Kimi K3: Open Frontier Intelligence

• URL: http://arxiv.org/abs/2607.24653v1
• Date: Mon, 27 Jul 2026 16:49:54 GMT
• Title: Kimi K3: Open Frontier Intelligence
• Abstract summary: We introduce Kimi K3, a 2.8T parameter Mixture-of-Experts model with 104 billion activated parameters, native vision capabilities, and a 1-million-token context window.<n>Kimi K3 is built on Kimi Delta Attention and Attention Residuals, which improve information flow across sequence length and model depth.<n>We show that Kimi K3 achieves frontier-level performance across long-horizon coding, agentic, knowledge, reasoning, and vision tasks.
• Score: 244.27422883681837
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: We introduce Kimi K3, a 2.8T parameter Mixture-of-Experts model with 104 billion activated parameters, native vision capabilities, and a 1-million-token context window. Kimi K3 is built on Kimi Delta Attention and Attention Residuals, which improve information flow across sequence length and model depth. Together with Stable LatentMoE, which effectively activates 16 of 896 routed experts per token, and refined training and data recipes, these advances yield an approximately 2.5x improvement in overall scaling efficiency over Kimi K2. Post-training highlights reinforcement learning across general, agentic, and coding domains and multiple reasoning-effort levels, enabling compositional generalization and robust long-horizon execution. At 2.8T scale, Kimi K3 is supported by infrastructure advances in multiple areas: algorithm-system co-design for KDA, perfectly balanced expert-parallel training with efficient memory management, million-token agentic RL with persistent rollout and sandbox states, and deployment innovations. Extensive evaluations show that Kimi K3 achieves frontier-level performance across long-horizon coding, agentic, knowledge, reasoning, and vision tasks. While its overall performance still trails the most powerful proprietary models, namely Claude Fable 5 and GPT-5.6 Sol, Kimi K3 consistently outperforms other open and proprietary models evaluated in our suite. We release the full Kimi K3 model weights to facilitate future research and accelerate the broader deployment and adoption of frontier intelligence.

Related papers

• Motif 3: Technical Report [26.937626894426916]
  We introduce a decoder-only Mixture-of-Experts language model with 314 billion total parameters and 13.2 billion activated per token.<n>MoE layer contains 384 routed experts, with eight selected per token.<n>We pretrain Motif 3 on approximately 12.5 trillion tokens spanning web documents, STEM, code, mathematics, multilingual content, and domain-specialized corpora.
  arXiv  Detail & Related papers  (2026-08-10T04:53:05Z)
• K-EXAONE 2.0 Technical Report [59.84003637088418]
  K-EXAONE 2.0 is an open-weight multilingual foundation model developed by LG AI Research.<n>Released under the Apache 2.0 license, K-EXAONE 2.0 enables the wider AI ecosystem to evaluate, deploy, adapt, and build upon it.
  arXiv  Detail & Related papers  (2026-08-05T06:41:48Z)
• Kimi K2.5: Visual Agentic Intelligence [236.05020914831047]
  Kimi K2.5 is an open-source multimodal agentic model designed to advance general agentic intelligence.<n>Building on this multimodal foundation, K2.5 introduces Agent Swarm, a self-directed parallel agent orchestration framework.
  arXiv  Detail & Related papers  (2026-02-02T16:17:38Z)
• Uni-MoE-2.0-Omni: Scaling Language-Centric Omnimodal Large Model with Advanced MoE, Training and Data [55.65426108082807]
  We build Uni-MoE-2.0- Omni from scratch through three core contributions.<n>It is capable of omnimodal understanding, as well as generating images, text, and speech.
  arXiv  Detail & Related papers  (2025-11-16T14:10:55Z)
• Kimi Linear: An Expressive, Efficient Attention Architecture [75.89211364086309]
  Kimi Linear is a hybrid linear attention architecture that, for the first time, outperforms full attention under fair comparisons.<n>At its core lies Kimi Delta Attention (KDA), an expressive linear attention module that extends Gated DeltaNet.<n>We show that Kimi Linear can be a drop-in replacement for full attention with superior performance and efficiency.
  arXiv  Detail & Related papers  (2025-10-30T16:59:43Z)
• Kimi K2: Open Agentic Intelligence [118.78600121345099]
  Kimi K2 is a large language model with 32 billion activated parameters and 1 trillion total parameters.<n>Based on MuonClip, K2 was pre-trained on 15.5 trillion tokens with zero loss spike.<n>Kimi K2 achieves state-of-the-art performance among open-source non-thinking models.
  arXiv  Detail & Related papers  (2025-07-28T05:35:43Z)
• Kimi-VL Technical Report [88.71933968931096]
  Kimi-VL is a vision-language model (VLM) that offers advanced multimodal reasoning, long-context understanding, and strong agent capabilities.<n>As a general-purpose VLM, Kimi-VL excels in multi-turn agent tasks (e.g., OSWorld), matching flagship models.<n>Building upon Kimi-VL, we introduce an advanced long-thinking variant: Kimi-VL-Thinking-2506.
  arXiv  Detail & Related papers  (2025-04-10T06:48:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.