SLAI T-Rex: Full-Parameter Post-training of the DeepSeek-V4 Family on Ascend SuperPOD

• URL: http://arxiv.org/abs/2607.20145v2
• Date: Thu, 30 Jul 2026 09:30:44 GMT
• Title: SLAI T-Rex: Full-Parameter Post-training of the DeepSeek-V4 Family on Ascend SuperPOD
• Abstract summary: This report presents an end-to-end optimization practice on the Ascend NPU SuperPOD.<n>We develop a hierarchical optimization framework spanning model-level parallelism, computation-communication orchestration, and low-level kernel execution.<n>The resulting system achieves 34.22% Model FLOPs Utilization (MFU) with a 2.93x improvement over the open-source baseline recipe.
• Score: 91.71050339479261
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Full-parameter post-training of trillion-parameter-scale MoE models introduces substantial system-level challenges for large-scale distributed training, including severe memory pressure, non-overlapped communication overhead, and inefficient kernel execution. While most large-scale LLM training systems are built around GPU-based clusters, this report presents an end-to-end optimization practice on the Ascend NPU SuperPOD. Using the DeepSeek-V4 model family as the target workload, we develop a hierarchical optimization framework spanning model-level parallelism, computation-communication orchestration, and low-level kernel execution. The resulting system achieves 34.22% Model FLOPs Utilization (MFU) with a 2.93x improvement over the open-source baseline recipe while maintaining training stability. Building on this optimized infrastructure, we further establish a CPT and SFT workflow for complex Operations Research (OR) tasks. We refer to the integrated framework as SLAI T-Rex. Using DeepSeek-V4-Flash, we develop OR-oriented CPT and SFT data pipelines that combine collected domain resources with solver-verified synthetic optimization documents. The resulting dataset contains 10K high-quality SFT samples spanning four task categories and three problem representations. The specialized model achieves the highest average zero-shot Pass@1 score among the evaluated models, reaching 71.81% and outperforming GPT-5.4-Mini and the base DeepSeek-V4-Flash model by 3.98 and 11.27 percentage points, respectively. Overall, this work demonstrates a full-stack pathway from efficient trillion-parameter model post-training on Ascend infra to domain-specialized Flash models for solver-grounded mathematical modeling, advancing frontier-model systems for complex reasoning.

Related papers

• Optimization Algorithms for Joint OFDM Waveform Design and RIS Configuration in 6G Networks: From Convex Relaxation to Foundation Models [1.9290392443571385]
  Seventy-eight joint OFDMRIS optimization works published between 2021 and 2026 are surveyed.<n>No standardized benchmark exists, and cross-paper comparisons remain infeasible.<n>Six open challenges emerge from synthesis-based benchmark deployment.
  arXiv  Detail & Related papers  (2026-06-30T08:34:46Z)
• Embodied-R1.5: Evolving Physical Intelligence via Embodied Foundation Models [63.96252564303143]
  Embodied-R1.5 is a unified Embodied Foundation Model (EFM) that integrates comprehensive embodied reasoning capabilities.<n>We build a large-scale data system of over 15B tokens, and design a multi-task balanced RL recipe.<n>With only 8B parameters, Embodied-R1.5 SOTA on 16 out of 24 embodied VLM benchmarks, surpassing leading models like Gemini-Robotics-ER-1.5 and GPT-5.4.
  arXiv  Detail & Related papers  (2026-06-09T18:07:50Z)
• AgentCPM-Explore: Realizing Long-Horizon Deep Exploration for Edge-Scale Agents [75.67445299298949]
  AgentCPM-Explore is a compact 4B agent model with high knowledge density and strong exploration capability.<n>We introduce a holistic training framework featuring parameter-space model fusion, reward signal denoising, and contextual information refinement.<n>AgentCPM-Explore achieves state-of-the-art (SOTA) performance among 4B-class models, matches or surpasses 8B-class SOTA models on four benchmarks, and even outperforms larger-scale models such as Claude-4.5-Sonnet or DeepSeek-v3.2 in five benchmarks.
  arXiv  Detail & Related papers  (2026-02-06T08:24:59Z)
• How to Train Your LLM Web Agent: A Statistical Diagnosis [96.86317871461834]
  We present the first statistically grounded study on compute allocation for LLM web-agent post-training.<n>Our approach uses a two-stage pipeline, training a Llama 3.1 8B student to imitate a Llama 3.3 70B teacher via supervised fine-tuning (SFT) and on-policy reinforcement learning.<n>Our results show that combining SFT with on-policy RL consistently outperforms either approach alone on both WorkArena and MiniWob++.
  arXiv  Detail & Related papers  (2025-07-05T17:12:33Z)
• Ring-lite: Scalable Reasoning via C3PO-Stabilized Reinforcement Learning for LLMs [51.21041884010009]
  Ring-lite is a Mixture-of-Experts (MoE)-based large language model optimized via reinforcement learning (RL)<n>Our approach matches the performance of state-of-the-art (SOTA) small-scale reasoning models on challenging benchmarks.
  arXiv  Detail & Related papers  (2025-06-17T17:12:34Z)
• MiniCPM4: Ultra-Efficient LLMs on End Devices [126.22958722174583]
  MiniCPM4 is a highly efficient large language model (LLM) designed explicitly for end-side devices.<n>We achieve this efficiency through systematic innovation in four key dimensions: model architecture, training data, training algorithms, and inference systems.
  arXiv  Detail & Related papers  (2025-06-09T16:16:50Z)
• From Drafts to Answers: Unlocking LLM Potential via Aggregation Fine-Tuning [31.95005389919542]
  Scaling data and model size has been proven effective for boosting the performance of large language models.<n>In this work, we introduce Aggregation Fine-Tuning (AFT), a supervised finetuning paradigm.<n> Empirical evaluations on benchmark datasets show that AFT-trained models substantially outperform standard SFT.
  arXiv  Detail & Related papers  (2025-01-21T04:11:59Z)
• Lightweight Deep Learning Framework for Accurate Particle Flow Energy Reconstruction [8.598010350935596]
  This paper systematically evaluates a deep learning reconstruction framework.<n>We design a hybrid loss function combining weighted mean squared with error structural similarity index.<n>We enhance the model's capability to capture cross-modaltemporal correlations and energy-displacement nonlinearities.
  arXiv  Detail & Related papers  (2024-10-08T11:49:18Z)
• A 4D Hybrid Algorithm to Scale Parallel Training to Thousands of GPUs [1.7481226034111275]
  This paper introduces a four-dimensional (4D) approach to optimize communication in parallel training. AxoNN surpasses Megatron-LM, a state-of-the-art framework, by a significant 26%. It achieves a significantly high 57% of the theoretical peak FLOP/s or 182 PFLOP/s in total.
  arXiv  Detail & Related papers  (2023-05-22T22:41:49Z)
• A Hybrid Tensor-Expert-Data Parallelism Approach to Optimize Mixture-of-Experts Training [13.346719319555943]
  Mixture-of-Experts (MoE) is a neural network architecture that adds sparsely activated expert blocks to a base model. Current distributed deep learning frameworks are limited in their ability to train high-quality MoE models with large base models. We present DeepSpeed-TED, a novel, three-dimensional, hybrid parallel algorithm that combines data, tensor, and expert parallelism.
  arXiv  Detail & Related papers  (2023-03-11T05:38:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.