INTELLECT-3: Technical Report

• URL: http://arxiv.org/abs/2512.16144v1
• Date: Thu, 18 Dec 2025 03:57:01 GMT
• Title: INTELLECT-3: Technical Report
• Authors: Prime Intellect Team, Mika Senghaas, Fares Obeid, Sami Jaghouar, William Brown, Jack Min Ong, Daniel Auras, Matej Sirovatka, Jannik Straube, Andrew Baker, Sebastian Müller, Justus Mattern, Manveer Basra, Aiman Ismail, Dominik Scherm, Cooper Miller, Ameen Patel, Simon Kirsten, Mario Sieg, Christian Reetz, Kemal Erdem, Vincent Weisser, Johannes Hagemann,
• Abstract summary: INTELLECT-3 is a Mixture-of-Experts model (12B active) trained with large-scale reinforcement learning.<n>We open-source the model together with the full infrastructure stack used to create it, including RL frameworks.<n>We introduce prime-rl, an open framework for large-scale asynchronous reinforcement learning.
• Score: 5.3998786788822
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present INTELLECT-3, a 106B-parameter Mixture-of-Experts model (12B active) trained with large-scale reinforcement learning on our end-to-end RL infrastructure stack. INTELLECT-3 achieves state of the art performance for its size across math, code, science and reasoning benchmarks, outperforming many larger frontier models. We open-source the model together with the full infrastructure stack used to create it, including RL frameworks, complete recipe, and a wide collection of environments, built with the verifiers library, for training and evaluation from our Environments Hub community platform. Built for this effort, we introduce prime-rl, an open framework for large-scale asynchronous reinforcement learning, which scales seamlessly from a single node to thousands of GPUs, and is tailored for agentic RL with first-class support for multi-turn interactions and tool use. Using this stack, we run both SFT and RL training on top of the GLM-4.5-Air-Base model, scaling RL training up to 512 H200s with high training efficiency.

Related papers

• DeepGen 1.0: A Lightweight Unified Multimodal Model for Advancing Image Generation and Editing [67.77471070868852]
  DeepGen 1.0 is a lightweight 5B unified model for image generation and editing.<n>It is trained on only 50M samples, surpassing the 80B HunyuanImage by 28% on WISE and the 27B Qwen-Image-Edit by 37% on UniREditBench.<n>By open-sourcing our training code, weights, and datasets, we provide an efficient, high-performance alternative to democratize unified multimodal research.
  arXiv  Detail & Related papers  (2026-02-12T17:44:24Z)
• RollArt: Scaling Agentic RL Training via Disaggregated Infrastructure [49.88201789074532]
  Agentic Reinforcement Learning (RL) enables Large Language Models (LLMs) to perform autonomous decision-making and long-term planning.<n>We present RollArc, a distributed system designed to maximize throughput for multi-task agentic RL on disaggregated infrastructure.
  arXiv  Detail & Related papers  (2025-12-27T11:14:23Z)
• Training Foundation Models on a Full-Stack AMD Platform: Compute, Networking, and System Design [26.12152103450326]
  We report on the first large-scale mixture-of-experts (MoE) pretraining study on pure AMD hardware.<n>We distill practical guidance for both systems and model design.<n>ZAYA1-base performance comparable to leading base models such as Qwen3-4B and Gemma3-12B at its scale and larger.
  arXiv  Detail & Related papers  (2025-11-21T10:44:02Z)
• Revolutionizing Reinforcement Learning Framework for Diffusion Large Language Models [49.911784762244814]
  TraceRL is a trajectory-aware reinforcement learning framework for diffusion language models (DLMs)<n>We derive a series of state-of-the-art diffusion language models, namely TraDo.<n>TraDo-8B-Instruct achieves relative accuracy improvements of 6.1% over Qwen2.5-7B-Instruct and 51.3% over Llama3.1-8B-Instruct on mathematical reasoning benchmarks.
  arXiv  Detail & Related papers  (2025-09-08T17:58:06Z)
• Ring-lite: Scalable Reasoning via C3PO-Stabilized Reinforcement Learning for LLMs [51.21041884010009]
  Ring-lite is a Mixture-of-Experts (MoE)-based large language model optimized via reinforcement learning (RL)<n>Our approach matches the performance of state-of-the-art (SOTA) small-scale reasoning models on challenging benchmarks.
  arXiv  Detail & Related papers  (2025-06-17T17:12:34Z)
• LlamaRL: A Distributed Asynchronous Reinforcement Learning Framework for Efficient Large-scale LLM Training [32.575669924032276]
  Reinforcement Learning (RL) has become the most effective post-training approach for improving the capabilities of Large Language Models (LLMs)<n>We present LlamaRL, a fully distributed, asynchronous RL framework optimized for efficient training of large-scale LLMs.
  arXiv  Detail & Related papers  (2025-05-29T22:14:15Z)
• INTELLECT-2: A Reasoning Model Trained Through Globally Decentralized Reinforcement Learning [2.2063836130393817]
  INTELLECT-2 is the first globally distributed reinforcement learning (RL) training run of a 32 billion parameter language model.<n>To enable a training run with this unique infrastructure, we built various components from scratch.<n>We open-source INTELLECT-2 along with all of our code and data, hoping to encourage and enable more open research in the field of decentralized training.
  arXiv  Detail & Related papers  (2025-05-12T07:24:33Z)
• Scaling Offline Model-Based RL via Jointly-Optimized World-Action Model Pretraining [49.730897226510095]
  We introduce JOWA: Jointly-Reinforced World-Action model, an offline model-based RL agent pretrained on Atari games with 6 billion tokens data.<n>Our largest agent, with 150 million parameters, 78.9% human-level performance on pretrained games using only 10% subsampled offline data, outperforming existing state-of-the-art large-scale offline RL baselines by 31.6% on averange.
  arXiv  Detail & Related papers  (2024-10-01T10:25:03Z)
• Bayesian Generational Population-Based Training [35.70338636901159]
  Population-Based Training (PBT) has led to impressive performance in several large scale settings. We introduce two new innovations in PBT-style methods. We show that these innovations lead to large performance gains.
  arXiv  Detail & Related papers  (2022-07-19T16:57:38Z)
• ElegantRL-Podracer: Scalable and Elastic Library for Cloud-Native Deep Reinforcement Learning [141.58588761593955]
  We present a library ElegantRL-podracer for cloud-native deep reinforcement learning. It efficiently supports millions of cores to carry out massively parallel training at multiple levels. At a low-level, each pod simulates agent-environment interactions in parallel by fully utilizing nearly 7,000 GPU cores in a single GPU.
  arXiv  Detail & Related papers  (2021-12-11T06:31:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.