ResMerge: Residual-based Spectral Merging of Large Language Models

• URL: http://arxiv.org/abs/2606.02252v1
• Date: Mon, 01 Jun 2026 13:42:45 GMT
• Title: ResMerge: Residual-based Spectral Merging of Large Language Models
• Abstract summary: We propose ResMerge, a residual-based spectral merging framework for RL experts.<n>Based on this observation, we propose ResMerge, a residual-based spectral merging framework for RL experts.
• Score: 52.65011413018957
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Model merging offers a training-free way to combine multiple post-trained expert models, but merging experts obtained through reinforcement learning (RL) remains challenging. Existing spectral merging methods often assume that leading singular directions contain the main task signal, while lower-energy residual components can be compressed, selected, or attenuated to reduce interference. We find that this assumption does not hold for RL task vectors: after decomposing each task vector into a leading spectral head and a residual component, both parts can independently recover substantial behavior knowledge, while exhibiting different merging properties. The head is highly concentrated and informative but more prone to sharp cross-expert conflicts, whereas the residual component is more dispersed and provides a more stable basis for aggregation. Based on this observation, we propose ResMerge, a residual-based spectral merging framework for RL experts. ResMerge first constructs a stable residual backbone with Spherical Residual Consensus Adaptation, which estimates a reliability-weighted consensus direction on the Frobenius sphere. It then reintroduces leading-head information through a Lightweight Head Correction module gated by positive cross-expert agreement. Experiments across multiple RL expert groups and capability domains show that ResMerge better preserves expert capabilities than representative task-vector and spectral merging baselines. The implementation of ResMerge is publicly available at https://github.com/sunyd0303-cpu/ResMerge-release.

Related papers

• ARISE-RL: Agentic Rubric-Grounded Iterative Self-Evolution with Reinforcement Learning [74.19730275669227]
  ARISE-RL is a novel full-cycle self-evolution framework for training open-ended agents.<n>We present Reward-Gated Self-Evolution Distillation (RG-SED), which distills a memory-augmented variant of the same policy back into itself.<n>We also present ECR-Bench, an expert-calibrated rubric benchmark suite covering single-tool deep research and multi-tool travel planning.
  arXiv  Detail & Related papers  (2026-09-01T10:54:13Z)
• Consolidating RLVR Capabilities Across Domains: A Deep Dive into Fusion Paradigms [50.28963615440865]
  We organize three fusion paradigms by the artefacts they reuse.<n>We compare all three using shared experts and data across model scales and a multi-domain benchmark suite.
  arXiv  Detail & Related papers  (2026-08-27T17:38:04Z)
• Share First, Route What Remains: A Unified Framework for Token-Adaptive MoE Computation [49.8459545291965]
  Mixture-of-experts (MoE) models have recently moved beyond routing a fixed number of complete experts.<n>We study this dependency by decomposing sparsely upcycled feed-forward experts into key-value channels.<n>We instantiate it in UniF-MoE, a unified framework for token-adaptive MoE computation.
  arXiv  Detail & Related papers  (2026-08-11T02:40:58Z)
• FedWeave: Rethinking the Unit of Specialization in Heterogeneous Federated MoE-LoRA [15.068543696107227]
  Federated PEFT enables LLMs to collaboratively adapt to decentralized private data without sharing raw examples.<n>Existing methods typically specialize at client granularity, implicitly assuming task-coherent clients.<n>We propose FedWeave, a framework that adopts asymmetric aggregation, separating expert aggregation from router optimization.
  arXiv  Detail & Related papers  (2026-07-29T08:47:58Z)
• RADD: Retrieval-Augmented Discrete Diffusion for Multi-Modal Knowledge Graph Completion [14.144464555343385]
  We propose a Retrieval-Augmented Discrete Diffusion framework to decouple retrieve and reranking for MMKGC.<n>A relation-aware multimodal KGE retriever serves as both global retriever and distillation teacher, while a conditional discrete denoiser performs shortlist-level entity-identity generation for reranking.
  arXiv  Detail & Related papers  (2026-04-28T14:21:03Z)
• Towards Adaptive Continual Model Merging via Manifold-Aware Expert Evolution [33.44708543696699]
  MADE-IT orchestrates expert management and activation by grounding intrinsic expert representations in manifold geometry.<n>Experiments demonstrate that MADE-IT consistently outperforms strong baselines in accuracy and robustness across long-horizon and shuffled task sequences.
  arXiv  Detail & Related papers  (2026-04-24T11:35:53Z)
• DR-MMSearchAgent: Deepening Reasoning in Multimodal Search Agents [24.61813749877376]
  Deepening Reasoning MMSearchAgent derives advantage signals from whole rollout trajectories in an entire batch.<n> differentiated gaussian rewards are employed to dynamically calibrate interaction tolerance.<n>Our method achieves state-of-the-art performance, outperforming the MMSearch-R1 by 8.4$%$ on FVQA-test.
  arXiv  Detail & Related papers  (2026-04-21T09:28:34Z)
• Does RL Expand the Capability Boundary of LLM Agents? A PASS@(k,T) Analysis [16.739617199649615]
  We introduce PASS@(k,T), a metric that separates capability expansion from efficiency improvement.<n>Our main finding is that, contrary to the static-reasoning result, tool-use RL genuinely enlarges the capability boundary.<n>Under matched training data, supervised fine-tuning regresses the boundary on the same compositional tasks, isolating self-directed exploration as the causal factor.
  arXiv  Detail & Related papers  (2026-04-16T11:06:19Z)
• A Replicate-and-Quantize Strategy for Plug-and-Play Load Balancing of Sparse Mixture-of-Experts LLMs [64.8510381475827]
  Sparse Mixture-of-Experts (SMoE) architectures are increasingly used to scale large language models efficiently.<n>SMoE models often suffer from severe load imbalance across experts, where a small subset of experts receives most tokens while others are underutilized.<n>We present a systematic analysis of expert routing during inference and identify three findings: (i) load imbalance persists and worsens with larger batch sizes, (ii) selection frequency does not reliably reflect expert importance, and (iii) overall expert workload and importance can be estimated using a small calibration set.
  arXiv  Detail & Related papers  (2026-02-23T15:11:16Z)
• Mixture of Ranks with Degradation-Aware Routing for One-Step Real-World Image Super-Resolution [76.66229730098759]
  In real-world image super-resolution (Real-ISR), existing approaches mainly rely on fine-tuning pre-trained diffusion models.<n>We propose a Mixture-of-Ranks (MoR) architecture for single-step image super-resolution.<n>We introduce a fine-grained expert partitioning strategy that treats each rank in LoRA as an independent expert.
  arXiv  Detail & Related papers  (2025-11-20T04:11:44Z)
• Real-Aware Residual Model Merging for Deepfake Detection [29.454462278073063]
  Deepfake generators evolve quickly, making exhaustive data collection and repeated retraining impractical.<n>We propose Real-aware Residual Model Merging (R$2$M), a training-free parameter-space merging framework.<n>R$2$M estimates a shared Real component via a low-rank factorization of task vectors, decomposes each specialist into a Real-aligned part and a Fake residual, denoises residuals with layerwise rank truncation, and aggregates them with per-task norm matching.
  arXiv  Detail & Related papers  (2025-09-29T07:10:03Z)
• Dropping Experts, Recombining Neurons: Retraining-Free Pruning for Sparse Mixture-of-Experts LLMs [54.95810313530111]
  DERN is a task-agnostic and retraining-free framework for expert pruning and reconstruction.<n>It improves performance by more than 5% on commonsense reasoning and MMLU benchmarks under 50% expert sparsity.
  arXiv  Detail & Related papers  (2025-09-12T16:09:39Z)
• Improving Retrieval-Augmented Generation through Multi-Agent Reinforcement Learning [88.55095746156428]
  Retrieval-augmented generation (RAG) is widely utilized to incorporate external knowledge into large language models.<n>A standard RAG pipeline consists of several components, such as query rewriting, document retrieval, document filtering, and answer generation.<n>We propose treating the complex RAG pipeline with multiple components as a multi-agent cooperative task, in which each component can be regarded as an RL agent.
  arXiv  Detail & Related papers  (2025-01-25T14:24:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.