Qwen3 Technical Report

• URL: http://arxiv.org/abs/2505.09388v1
• Date: Wed, 14 May 2025 13:41:34 GMT
• Title: Qwen3 Technical Report
• Authors: An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jing Zhou, Jingren Zhou, Junyang Lin, Kai Dang, Keqin Bao, Kexin Yang, Le Yu, Lianghao Deng, Mei Li, Mingfeng Xue, Mingze Li, Pei Zhang, Peng Wang, Qin Zhu, Rui Men, Ruize Gao, Shixuan Liu, Shuang Luo, Tianhao Li, Tianyi Tang, Wenbiao Yin, Xingzhang Ren, Xinyu Wang, Xinyu Zhang, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yinger Zhang, Yu Wan, Yuqiong Liu, Zekun Wang, Zeyu Cui, Zhenru Zhang, Zhipeng Zhou, Zihan Qiu,
• Abstract summary: We present Qwen3, the latest version of the Qwen model family.<n>Qwen3 comprises a series of large language models (LLMs) designed to advance performance, efficiency, and multilingual capabilities.
• Score: 137.96804244102205
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this work, we present Qwen3, the latest version of the Qwen model family. Qwen3 comprises a series of large language models (LLMs) designed to advance performance, efficiency, and multilingual capabilities. The Qwen3 series includes models of both dense and Mixture-of-Expert (MoE) architectures, with parameter scales ranging from 0.6 to 235 billion. A key innovation in Qwen3 is the integration of thinking mode (for complex, multi-step reasoning) and non-thinking mode (for rapid, context-driven responses) into a unified framework. This eliminates the need to switch between different models--such as chat-optimized models (e.g., GPT-4o) and dedicated reasoning models (e.g., QwQ-32B)--and enables dynamic mode switching based on user queries or chat templates. Meanwhile, Qwen3 introduces a thinking budget mechanism, allowing users to allocate computational resources adaptively during inference, thereby balancing latency and performance based on task complexity. Moreover, by leveraging the knowledge from the flagship models, we significantly reduce the computational resources required to build smaller-scale models, while ensuring their highly competitive performance. Empirical evaluations demonstrate that Qwen3 achieves state-of-the-art results across diverse benchmarks, including tasks in code generation, mathematical reasoning, agent tasks, etc., competitive against larger MoE models and proprietary models. Compared to its predecessor Qwen2.5, Qwen3 expands multilingual support from 29 to 119 languages and dialects, enhancing global accessibility through improved cross-lingual understanding and generation capabilities. To facilitate reproducibility and community-driven research and development, all Qwen3 models are publicly accessible under Apache 2.0.

Related papers

• Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Models [90.54780244175511]
  We introduce the Qwen3 Embedding series, a significant advancement over its predecessor, the GTE-Qwen series.<n>The Qwen3 Embedding series offers a spectrum of model sizes for both embedding and reranking tasks.<n>The Qwen3 Embedding series achieves state-of-the-art results across diverse benchmarks.
  arXiv  Detail & Related papers  (2025-06-05T15:49:48Z)
• An Empirical Study of Qwen3 Quantization [30.214896404069677]
  Low-bit quantization presents a promising solution, yet its impact on Qwen3's performance remains underexplored.<n>We rigorously assess 5 existing classic post-training quantization techniques applied to Qwen3, spanning bit-widths from 1 to 8 bits.<n>Our findings reveal that while Qwen3 maintains competitive performance at moderate bit-widths, it experiences notable degradation in linguistic tasks under ultra-low precision.
  arXiv  Detail & Related papers  (2025-05-04T18:43:44Z)
• Qwen2.5 Technical Report [122.13958993185952]
  We introduce Qwen2.5, a comprehensive series of large language models (LLMs) designed to meet diverse needs.<n>Compared to previous iterations, Qwen 2.5 has been significantly improved during both the pre-training and post-training stages.<n>Open-weight offerings include base and instruction-tuned models, with quantized versions available.<n>For hosted solutions, the proprietary models currently include two mixture-of-experts (MoE) variants: Qwen2.5-Turbo and Qwen2.5-Plus.
  arXiv  Detail & Related papers  (2024-12-19T17:56:09Z)
• Qwen2 Technical Report [141.0766756297144]
  This report introduces the Qwen2 series, the latest addition to our large language models and large multimodal models. Qwen2 surpasses most prior open-weight models, including its predecessor Qwen1.5, and exhibits competitive performance relative to proprietary models. Qwen2 demonstrates robust multilingual capabilities, proficient in approximately 30 languages, spanning English, Chinese, Spanish, French, German, Arabic, Russian, Korean, Japanese, Thai, Vietnamese, and more.
  arXiv  Detail & Related papers  (2024-07-15T12:35:42Z)
• In-Context Language Learning: Architectures and Algorithms [73.93205821154605]
  We study ICL through the lens of a new family of model problems we term in context language learning (ICLL) We evaluate a diverse set of neural sequence models on regular ICLL tasks.
  arXiv  Detail & Related papers  (2024-01-23T18:59:21Z)
• Qwen Technical Report [132.54304067403922]
  We introduce Qwen, the first installment of our large language model series. Qwen is the base pretrained language models, and Qwen-Chat, the chat models finetuned with human alignment techniques. We have also developed coding-specialized models, Code-Qwen and Code-Qwen-Chat, as well as mathematics-focused models, Math-Qwen-Chat.
  arXiv  Detail & Related papers  (2023-09-28T17:07:49Z)
• Large Language Models Are Also Good Prototypical Commonsense Reasoners [11.108562540123387]
  Traditional fine-tuning approaches can be resource-intensive and potentially compromise a model's generalization capacity. We draw inspiration from the outputs of large models for tailored tasks and semi-automatically developed a set of novel prompts. With better designed prompts we can achieve the new state-of-art(SOTA) on the ProtoQA leaderboard.
  arXiv  Detail & Related papers  (2023-09-22T20:07:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.