Solar Open 2 Technical Report

• URL: http://arxiv.org/abs/2607.20062v2
• Date: Fri, 24 Jul 2026 02:46:10 GMT
• Title: Solar Open 2 Technical Report
• Abstract summary: We present Solar Open 2, a 250B-A15B Mixture-of-Experts language model built for long-horizon agentic tasks.<n>To hold entire agent trajectories in a single context, Solar Open 2 reaches a 1M-token window.<n>We train twelve domain specialists across purpose-built scenarios, then consolidate them into a single model.
• Score: 96.21161740206544
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present Solar Open 2, a 250B-A15B Mixture-of-Experts language model built for long-horizon agentic tasks, scaled up from Solar Open 1 (Solar Open 100B). To hold entire agent trajectories in a single context, Solar Open 2 reaches a 1M-token window through a hybrid attention stack that interleaves one softmax layer among every three linear-attention layers, using no positional encoding and a gated delta rule extended to negative eigenvalues. To train at this scale under a fixed compute budget, we make training efficient in two ways: a stronger starting point, and higher-value data. For the starting point, we initialize Solar Open 2 from Solar Open 1, transferring the 5.69B-parameter shared skeleton that survives the architectural change and learning everything else through full pre-training. For the data, we curate for value per token: quality- and rarity-aware data curation and mixture-ratio optimization refine a 20T pool into a 10T mixture that, at equal token budget, outperforms the Solar Open 1 recipe. To build its agent skills, we train twelve domain specialists across purpose-built scenarios, then consolidate them into a single model by Multi-teacher On-Policy Distillation (MOPD). Against comparably sized open-weight models on English benchmarks, Solar Open 2 leads on MMLU-Pro, LiveCodeBench, and the APEX-Agents agentic suite, and stays competitive with the strongest (DeepSeek-V4-Flash and MiMo-V2.5) elsewhere. On Korean benchmarks, Solar Open 2 records the highest average of any model compared, including fast-tier closed APIs, and on Ko-GDPval, an in-house Korean officework-agent benchmark, it is competitive with DeepSeek-V4-Pro (1.6T) at less than a sixth of its size.

Related papers

• Opt.Gear Technical Report [3.573729952898132]
  OptGear is a foundation model designed for efficient on-device deployment, real-tim inference, and strong task capability.<n>It includes a dense model (1M, 270M, and 1B) with a context length of 64K.<n>OptGear-1M is the first generative language model to achieve 20 TPS with W4A32 quantization on the ARM Cortex-M7 CPU.
  arXiv  Detail & Related papers  (2026-08-02T06:43:25Z)
• Nemotron 3 Ultra: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoning [269.10652784788374]
  Nemotron 3 Ultra is a 550 billion total and 55 billion active parameter Mixture-of-Experts Hybrid Mamba-Attention language model.<n>We pre-trained Nemotron 3 Ultra on 20 trillion text tokens, then extended the context length to 1M tokens, and post-trained using Supervised Fine Tuning (SFT), Reinforcement Learning (RL), and Multi-teacher On-Policy Distillation (MOPD)
  arXiv  Detail & Related papers  (2026-06-12T22:56:12Z)
• MolmoAct2: Action Reasoning Models for Real-world Deployment [67.6315757474802]
  MolmoAct2 is a fully open action reasoning model built for practical deployment.<n>We introduce MolmoER, a VLM backbone specialized for spatial and embodied reasoning.<n>We release three new datasets spanning low-to-medium cost platforms.
  arXiv  Detail & Related papers  (2026-05-04T17:51:21Z)
• Solar Open Technical Report [65.93022715874504]
  Solar Open demonstrates a systematic methodology for building competitive LLMs.<n>We synthesize 4.5T tokens of high-quality, domain-specific, and RL-oriented data.<n>We apply our proposed framework SnapPO for efficient optimization.
  arXiv  Detail & Related papers  (2026-01-11T18:33:09Z)
• Uni-MoE-2.0-Omni: Scaling Language-Centric Omnimodal Large Model with Advanced MoE, Training and Data [55.65426108082807]
  We build Uni-MoE-2.0- Omni from scratch through three core contributions.<n>It is capable of omnimodal understanding, as well as generating images, text, and speech.
  arXiv  Detail & Related papers  (2025-11-16T14:10:55Z)
• APIGen-MT: Agentic Pipeline for Multi-Turn Data Generation via Simulated Agent-Human Interplay [86.01901238059261]
  APIGen-MT is a framework that generates verifiable and diverse multi-turn agent data.<n>We train a family of models -- the xLAM-2-fc-r series with sizes ranging from 1B to 70B parameters.<n>Our models outperform frontier models such as GPT-4o and Claude 3.5 on $tau$-bench and BFCL benchmarks.
  arXiv  Detail & Related papers  (2025-04-04T17:13:57Z)
• H2O-Danube-1.8B Technical Report [2.6856284636402106]
  We present H2O-Danube, a series of small 1.8B language models. H2O-Danube2-1.8B achieves the top ranking on Open LLM Leaderboard for all models below the 2B parameter range.
  arXiv  Detail & Related papers  (2024-01-30T08:45:08Z)
• Gaussian Process Regression for Probabilistic Short-term Solar Output Forecast [1.7205106391379026]
  This paper proposes a probabilistic framework to predict short-term PV output taking into account the uncertainty of weather. We make use of datasets that comprise of power output and meteorological data such as irradiance, temperature, zenith, and azimuth. We validate our method with five solar generation plants in different locations and compare the results with existing methodologies.
  arXiv  Detail & Related papers  (2020-02-23T15:54:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.