HyperCLOVA X THINK Technical Report

• URL: http://arxiv.org/abs/2506.22403v2
• Date: Tue, 01 Jul 2025 13:39:25 GMT
• Title: HyperCLOVA X THINK Technical Report
• Authors: NAVER Cloud HyperCLOVA X Team,
• Abstract summary: We introduce HyperCLOVA X THINK, the first reasoning-focused large language model in the HyperCLOVA X family.<n>It pre-trained on roughly $6$ trillion high-quality Korean, and English tokens, augmented with targeted synthetic Korean data.<n>It delivers competitive performance against similarly sized models on Korea-focused benchmarks.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We introduce HyperCLOVA X THINK, the first reasoning-focused large language model in the HyperCLOVA X family, pre-trained on roughly $6$ trillion high-quality Korean, and English tokens, augmented with targeted synthetic Korean data. It was implemented as a compute-memory-balanced Peri-LN Transformer scaled with $\mu$P, pre-trained through a three-stage curriculum that expands the context window to $128$K tokens, and post-trained via supervised fine-tuning with Reinforcement Learning from Verifiable Rewards supports both detailed rationale and concise-answer modes. It delivers competitive performance against similarly sized models on Korea-focused benchmarks such as KMMLU, CSAT, KoBALT-700, HAERAE-1.0, and KoBigBench, while preserving robust bilingual consistency and translation quality. In addition, a vision-augmented variant matches or exceeds GPT-4.1 on the KCSAT STEM benchmark, all of which are achieved with substantially lower training compute than existing models of similar sizes. We also present a pruning and distillation technique that will soon be applied to HyperCLOVA X THINK for an open-source and business-friendly foundation model. Altogether, these capabilities position HyperCLOVA X THINK as a robust foundation for Korean AI innovation and a valuable resource for the global research community.

Related papers

• A.X K1 Technical Report [24.287781467694227]
  A.X K1 is a Mixture-of-Experts (MoE) language model trained from scratch.<n>A.X K1 is pre-trained on a corpus of approximately 10T tokens, curated by a multi-stage data processing pipeline.<n>A.X K1 supports explicitly controllable reasoning to facilitate scalable deployment across diverse real-world scenarios.
  arXiv  Detail & Related papers  (2026-01-14T06:11:17Z)
• K-EXAONE Technical Report [76.23621600385238]
  K-EXAONE is a large-scale multilingual language model developed by LG AI Research.<n>It supports a 256K-token context window and covers six languages: Korean, English, Spanish, German, Japanese, and Vietnamese.<n>We evaluate K-EXAONE on a comprehensive benchmark suite spanning reasoning, agentic, general, Korean, and multilingual abilities.
  arXiv  Detail & Related papers  (2026-01-05T02:30:59Z)
• NMIXX: Domain-Adapted Neural Embeddings for Cross-Lingual eXploration of Finance [24.697311579081983]
  General-purpose sentence embedding models often struggle to capture specialized financial semantics.<n>We introduce NMIXX, a suite of cross-lingual embedding models fine-tuned with 18.8K high-confidence triplets.<n>We also release KorFinSTS, a benchmark spanning news, disclosures, research reports, and regulations.
  arXiv  Detail & Related papers  (2025-07-13T12:14:57Z)
• Hunyuan-TurboS: Advancing Large Language Models through Mamba-Transformer Synergy and Adaptive Chain-of-Thought [196.74837065805488]
  Hunyuan-TurboS is a large hybrid Transformer-Mamba Mixture of Experts model.<n>It balances high performance and efficiency, offering substantial capabilities at lower inference costs.
  arXiv  Detail & Related papers  (2025-05-21T12:11:53Z)
• HyperCLOVA X Technical Report [119.94633129762133]
  We introduce HyperCLOVA X, a family of large language models (LLMs) tailored to the Korean language and culture. HyperCLOVA X was trained on a balanced mix of Korean, English, and code data, followed by instruction-tuning with high-quality human-annotated datasets. The model is evaluated across various benchmarks, including comprehensive reasoning, knowledge, commonsense, factuality, coding, math, chatting, instruction-following, and harmlessness, in both Korean and English.
  arXiv  Detail & Related papers  (2024-04-02T13:48:49Z)
• Efficient and Effective Vocabulary Expansion Towards Multilingual Large Language Models [9.359647125218359]
  This report introduces textttEEVE-Korean-v1.0, a Korean adaptation of large language models. Our method can significantly boost non-English proficiency within just 2 billion tokens.
  arXiv  Detail & Related papers  (2024-02-22T17:12:39Z)
• Beyond English-Centric Bitexts for Better Multilingual Language Representation Learning [99.42850643947439]
  We show that going beyond English-centric bitexts, coupled with a novel sampling strategy, substantially boosts performance across model sizes. Our XY-LENT XL variant outperforms XLM-RXXL and exhibits competitive performance with mT5 XXL while being 5x and 6x smaller respectively.
  arXiv  Detail & Related papers  (2022-10-26T17:16:52Z)
• Zemi: Learning Zero-Shot Semi-Parametric Language Models from Multiple Tasks [77.90900650816046]
  We introduce $textZemi$, a zero-shot semi-parametric language model. We train $textZemi$ with a novel semi-parametric multitask prompted training paradigm. Specifically, we augment the multitask training and zero-shot evaluation with retrieval from a large-scale task-agnostic unlabeled corpus.
  arXiv  Detail & Related papers  (2022-10-01T04:08:50Z)
• DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing [117.41016786835452]
  This paper presents a new pre-trained language model, DeBERTaV3, which improves the original DeBERTa model. vanilla embedding sharing in ELECTRA hurts training efficiency and model performance. We propose a new gradient-disentangled embedding sharing method that avoids the tug-of-war dynamics.
  arXiv  Detail & Related papers  (2021-11-18T06:48:00Z)
• Improving Neural Machine Translation by Bidirectional Training [85.64797317290349]
  We present a simple and effective pretraining strategy -- bidirectional training (BiT) for neural machine translation. Specifically, we bidirectionally update the model parameters at the early stage and then tune the model normally. Experimental results show that BiT pushes the SOTA neural machine translation performance across 15 translation tasks on 8 language pairs significantly higher.
  arXiv  Detail & Related papers  (2021-09-16T07:58:33Z)
• What Changes Can Large-scale Language Models Bring? Intensive Study on HyperCLOVA: Billions-scale Korean Generative Pretrained Transformers [16.596023525331862]
  GPT-3 shows remarkable in-context learning ability of large-scale language models (LMs) trained on hundreds of billion scale data. We introduce HyperCLOVA, a Korean variant of 82B GPT-3 trained on a Korean-centric corpus of 560B tokens. We show the performance benefits of prompt-based learning and demonstrate how it can be integrated into the prompt engineering pipeline.
  arXiv  Detail & Related papers  (2021-09-10T03:32:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.