Consolidating RLVR Capabilities Across Domains: A Deep Dive into Fusion Paradigms

• URL: http://arxiv.org/abs/2608.27409v1
• Date: Thu, 27 Aug 2026 17:38:04 GMT
• Title: Consolidating RLVR Capabilities Across Domains: A Deep Dive into Fusion Paradigms
• Abstract summary: We organize three fusion paradigms by the artefacts they reuse.<n>We compare all three using shared experts and data across model scales and a multi-domain benchmark suite.
• Score: 50.28963615440865
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Reinforcement learning with verifiable rewards (RLVR) improves specific capabilities of large language models, but covering multiple capabilities often involves training separate domain experts and subsequently consolidating them. We organize three fusion paradigms by the artefacts they reuse: Merge combines expert task vectors, Mix RL pools their datasets, and multi-teacher on-policy distillation (MOPD) uses both. Because they have largely been studied in isolation, how they compare and how to choose among them remain unclear. We compare all three using shared experts and data across model scales and a multi-domain benchmark suite. Although their average performance differs by at most 1.4 points, the gap reaches 8.6 points on a single benchmark, with domain-level variation tracking cross-domain relations visible in task-vector geometry. Training dynamics expose distinct constraints: Mix RL depends on domain mixture proportions, MOPD remains bounded by its teachers, and Merge compresses all expert updates into one. All three improve single-sample accuracy without measurable gains in solution coverage or losses in held-out capabilities. These results yield a practical guideline: use Merge when experts already exist and cheap fusion is paramount; Mix RL when training a unified model without experts, with domain proportions adjusted for cross-domain transfer; and MOPD when preserving domain-specific gains matters more than surpassing teachers or minimizing end-to-end cost.

Related papers

• Co-RL: Unsupervised Reasoning Emerges from Diverse Cohort in Multi-agent RL [55.05743310621117]
  Co-RL is a framework in which multiple decoupled models, sharing no parameters, are simultaneously optimized through RL using rewards derived from their peers.<n>We show that increasing cohort diversity, through heterogeneous model families, sizes, and rephrased training samples, reduces the correlated errors that drive self-reinforcing feedback loops.
  arXiv  Detail & Related papers  (2026-08-18T01:16:02Z)
• Multi-Domain Learning with Global Expert Mapping [102.62297074508147]
  Mixture-of-Experts (MoE) models provide a scalable solution by routing inputs to specializedworks (experts)<n>We propose GEM (Global Expert Mapping), a planner-compiler framework that replaces the learned router with a global scheduler.<n>Our planner, based on linear programming relaxation, computes a fractional assignment of datasets to experts, while the compiler applies hierarchical rounding to convert this soft plan into a deterministic, capacity-aware mapping.<n>Experiments show that GEM-DINO achieves state-of-the-art performance on the UODB benchmark, with notable gains on underrepresented datasets and solves task interference in few
  arXiv  Detail & Related papers  (2026-04-20T21:09:34Z)
• To Mix or To Merge: Toward Multi-Domain Reinforcement Learning for Large Language Models [36.65680320664014]
  Reinforcement Learning with Verifiable Rewards (RLVR) plays a key role in stimulating the explicit reasoning capability of Large Language Models (LLMs)<n>We design extensive qualitative and quantitative experiments using open-source datasets.<n>We find the RLVR across domains exhibits few mutual interferences, and reasoning-intensive domains demonstrate mutually synergistic effects.
  arXiv  Detail & Related papers  (2026-02-13T03:25:13Z)
• Token-Level LLM Collaboration via FusionRoute [60.72307345997823]
  FusionRoute is a token-level multi-LLM collaboration framework.<n>It selects the most suitable expert at each decoding step and contributes a complementary logit that refines or corrects the selected expert's next-token distribution.<n>It outperforms both sequence- and token-level collaboration, model merging, and direct fine-tuning.
  arXiv  Detail & Related papers  (2026-01-08T16:53:16Z)
• DoReMi: A Domain-Representation Mixture Framework for Generalizable 3D Understanding [10.259254902492978]
  DoReMi is a Mixture-of-Experts (MoE) framework that jointly models Domain-aware Experts branch and a unified Representation branch.<n>DoReMi achieves 80.1% mIoU on ScanNet Val and 77.2% mIoU on S3DIS, demonstrating competitive or superior performance compared to existing approaches.
  arXiv  Detail & Related papers  (2025-11-14T12:32:45Z)
• Mixture of Thoughts: Learning to Aggregate What Experts Think, Not Just What They Say [4.273730624882391]
  Mixture of Thoughts (MoT) is a simple method for latent-level collaboration among heterogeneous experts under a global routing scheme.<n>MoT surpasses the current routing and aggregation-based state-of-the-art, Avengers, by $+0.38%$ and $+2.92%$, respectively.
  arXiv  Detail & Related papers  (2025-09-25T13:50:09Z)
• HDEE: Heterogeneous Domain Expert Ensemble [0.3641292357963815]
  Several studies aim to reduce the dependency on centralization by reducing the communication overhead of training dense models.<n>Taking this idea of reducing communication overhead to a natural extreme, has been shown to outperform large dense models trained in traditional centralized settings.<n>We show that the heterogeneous ensembles achieve the lowest perplexity scores in $20$ out of the $21$ data domains used in the evaluation.
  arXiv  Detail & Related papers  (2025-02-26T18:30:49Z)
• META: Mimicking Embedding via oThers' Aggregation for Generalizable Person Re-identification [68.39849081353704]
  Domain generalizable (DG) person re-identification (ReID) aims to test across unseen domains without access to the target domain data at training time. This paper presents a new approach called Mimicking Embedding via oThers' Aggregation (META) for DG ReID.
  arXiv  Detail & Related papers  (2021-12-16T08:06:50Z)
• Domain Adaptive Ensemble Learning [141.98192460069765]
  We propose a unified framework termed domain adaptive ensemble learning (DAEL) to address both problems. Experiments on three multi-source UDA and two DG datasets show that DAEL improves the state of the art on both problems, often by significant margins.
  arXiv  Detail & Related papers  (2020-03-16T16:54:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.