EXAONE Deep: Reasoning Enhanced Language Models

• URL: http://arxiv.org/abs/2503.12524v2
• Date: Wed, 19 Mar 2025 07:09:24 GMT
• Title: EXAONE Deep: Reasoning Enhanced Language Models
• Authors: LG AI Research, Kyunghoon Bae, Eunbi Choi, Kibong Choi, Stanley Jungkyu Choi, Yemuk Choi, Seokhee Hong, Junwon Hwang, Hyojin Jeon, Kijeong Jeon, Gerrard Jeongwon Jo, Hyunjik Jo, Jiyeon Jung, Hyosang Kim, Joonkee Kim, Seonghwan Kim, Soyeon Kim, Sunkyoung Kim, Yireun Kim, Yongil Kim, Youchul Kim, Edward Hwayoung Lee, Haeju Lee, Honglak Lee, Jinsik Lee, Kyungmin Lee, Sangha Park, Yongmin Park, Sihoon Yang, Heuiyeen Yeen, Sihyuk Yi, Hyeongu Yun,
• Abstract summary: We present EXAONE Deep series, which exhibits superior capabilities in various reasoning tasks.<n>We train our models mainly on the reasoning-specialized dataset that incorporates long streams of thought processes.
• Score: 35.326172288018505
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: We present EXAONE Deep series, which exhibits superior capabilities in various reasoning tasks, including math and coding benchmarks. We train our models mainly on the reasoning-specialized dataset that incorporates long streams of thought processes. Evaluation results show that our smaller models, EXAONE Deep 2.4B and 7.8B, outperform other models of comparable size, while the largest model, EXAONE Deep 32B, demonstrates competitive performance against leading open-weight models. All EXAONE Deep models are openly available for research purposes and can be downloaded from https://huggingface.co/LGAI-EXAONE

Related papers

• EXAONE 4.0: Unified Large Language Models Integrating Non-reasoning and Reasoning Modes [42.31740630042654]
  EXAONE 4.0 integrates a Non-reasoning mode and a Reasoning mode to achieve both the excellent usability of EXAONE 3.5 and the advanced reasoning abilities of EXAONE Deep.<n>The EXAONE 4.0 model series consists of two sizes: a mid-size 32B model optimized for high performance, and a small-size 1.2B model designed for on-device applications.
  arXiv  Detail & Related papers  (2025-07-15T15:24:51Z)
• Light-R1: Curriculum SFT, DPO and RL for Long COT from Scratch and Beyond [14.372747932761754]
  This paper introduces Light-R1, an open-source suite for training long reasoning models. Our curriculum training progressively increases data difficulty, combined with multi-staged post-training. Our final Light-R1-14B-DS achieves SOTA performance among 14B models in math, with AIME24 & 25 scores of 74.0 and 60.2 respectively.
  arXiv  Detail & Related papers  (2025-03-13T15:29:22Z)
• Finedeep: Mitigating Sparse Activation in Dense LLMs via Multi-Layer Fine-Grained Experts [82.74439280067492]
  Finedeep is a deep-layered fine-grained expert architecture for dense models.<n>Our framework partitions the feed-forward neural network layers of traditional dense models into small experts.<n>A novel routing mechanism is proposed to determine each expert's contribution.
  arXiv  Detail & Related papers  (2025-02-18T15:09:58Z)
• Quantifying the Capability Boundary of DeepSeek Models: An Application-Driven Performance Analysis [12.79754082920348]
  We conduct a systematic evaluation of the DeepSeek-V3, DeepSeek-R1, DeepSeek-R1-Distill-Qwen series, DeepSeek-R1-Distill-Llama series, and the reasoning model QwQ-32B. We quantify the capability boundary of DeepSeek models through performance tier classifications. We develop a model selection handbook that clearly illustrates the relation among models, their capabilities and practical applications.
  arXiv  Detail & Related papers  (2025-02-16T15:29:58Z)
• EXAONE 3.5: Series of Large Language Models for Real-world Use Cases [35.04562823885241]
  The EXAONE 3.5 language models are offered in three configurations: 32B, 7.8B, and 2.4B.<n>For commercial use, please reach out to the official contact point of LG AI Research.
  arXiv  Detail & Related papers  (2024-12-06T08:53:46Z)
• A Collaborative Ensemble Framework for CTR Prediction [73.59868761656317]
  We propose a novel framework, Collaborative Ensemble Training Network (CETNet), to leverage multiple distinct models. Unlike naive model scaling, our approach emphasizes diversity and collaboration through collaborative learning. We validate our framework on three public datasets and a large-scale industrial dataset from Meta.
  arXiv  Detail & Related papers  (2024-11-20T20:38:56Z)
• What Matters for Model Merging at Scale? [94.26607564817786]
  Model merging aims to combine multiple expert models into a more capable single model. Previous studies have primarily focused on merging a few small models. This study systematically evaluates the utility of model merging at scale.
  arXiv  Detail & Related papers  (2024-10-04T17:17:19Z)
• EXAONE 3.0 7.8B Instruction Tuned Language Model [41.95996640625627]
  EXAONE 3.0 instruction-tuned language model is the first open model in the family of Large Language Models (LLMs) EXAONE 3.0 demonstrates highly competitive real-world performance with instruction-following capability against other state-of-the-art open models of similar size. Our comparative analysis shows that EXAONE 3.0 excels particularly in Korean, while achieving compelling performance across general tasks and complex reasoning.
  arXiv  Detail & Related papers  (2024-08-07T04:38:38Z)
• Using Game Play to Investigate Multimodal and Conversational Grounding in Large Multimodal Models [14.878276985702685]
  In this paper, we bring a recently developed evaluation paradigm from text models to multimodal models. We define games that challenge a model's capability to represent a situation from visual information and align such representations through dialogue. We find that the largest closed models perform rather well on the games that we define, while even the best open-weight models struggle with them.
  arXiv  Detail & Related papers  (2024-06-20T06:56:19Z)
• How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resources [117.6496550359768]
  This work explores recent advances in instruction-tuning language models on a range of open instruction-following datasets. We provide a large set of instruction-tuned models from 6.7B to 65B parameters in size, trained on 12 instruction datasets. We evaluate them on their factual knowledge, reasoning, multilinguality, coding, and open-ended instruction following abilities.
  arXiv  Detail & Related papers  (2023-06-07T19:59:23Z)
• Efficient Large Scale Language Modeling with Mixtures of Experts [61.45159383372181]
  Mixture of Experts layers (MoEs) enable efficient scaling of language models through conditional computation. This paper presents a detailed empirical study of how autoregressive MoE language models scale in comparison with dense models in a wide range of settings.
  arXiv  Detail & Related papers  (2021-12-20T17:05:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.