A Local Perturbation Theory for Cross-Domain Interference and Recovery in Multi-Domain RL

• URL: http://arxiv.org/abs/2606.02398v1
• Date: Mon, 01 Jun 2026 15:44:56 GMT
• Title: A Local Perturbation Theory for Cross-Domain Interference and Recovery in Multi-Domain RL
• Abstract summary: We show that single-domain RL produces sparse, small-magnitude parameter edits with weak overlap among top-changed neurons.<n>We prove under a local model of multi-domain RL that later-domain training harms an earlier domain mainly through a second-order damage term.
• Score: 49.589717630468876
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Reinforcement learning (RL) post-training improves large language models (LLMs) on individual domains such as mathematical reasoning, code generation, question answering, and creative writing (CW), but training on one domain often degrades performance on others. Existing explanations based on catastrophic forgetting or global gradient conflict are incomplete: substantial interference can occur even when full-model gradients are nearly orthogonal. We show that single-domain RL produces sparse, small-magnitude parameter edits with weak overlap among top-changed neurons, while different domains still share substantial active computation routes on which update directions determine whether they act synergistically or conflict. Guided by this observation, we prove under a local perturbation model of multi-domain RL that later-domain training harms an earlier domain mainly through a second-order damage term, which under the observed sparse route structure concentrates in a low-dimensional shared conflict subspace. Moreover, a short domain refresh contracts the harmful component on this subspace, enabling selective recovery with limited collateral damage. Consistent with the theory, a brief Re-Math refresh after Code $\rightarrow$ Math $\rightarrow$ QA $\rightarrow$ CW recovers Math from 57.66 to 66.04 while largely preserving performance on the other domains, yielding the best average score of 66.39. Beyond refresh, a training-free rollback on a sparse proxy conflict coordinate set for the Math-QA pair partially restores Math, providing direct proxy-level evidence for localized damage. These results provide a localized mechanistic account of interference and recovery in multi-domain RL.

Related papers

• One Step, One Lead: Mitigating Higher-Order Interference in Multi-Domain Reinforcement Learning via Cross-Step Control [62.82372407840088]
  We show that same-point domain gradients may remain nearly even when consecutive realized updates partially reverse one another in output space.<n>We propose OSOL, which designates a focus domain at each iteration, to rank token-level rebound risk.<n>OSOL reaches a domain-macro average of 0.4822, improving by 5.7% over the strongest compared baseline, without explicit higher-order differentiation.
  arXiv  Detail & Related papers  (2026-09-06T08:32:59Z)
• Orthogonal Knowledge Refreshing for Domain-Incremental Object Detection [37.885906679929796]
  Domain-incremental object detection requires models to continually adapt to new domains while preserving prior knowledge.<n>We propose Orthogonal Knowledge Refreshing (OKR), a simple yet effective framework for DIOD.<n>OKR incrementally constructs independent domain-specific subspaces via dedicated low-rank branches for each domain.
  arXiv  Detail & Related papers  (2026-07-19T17:02:15Z)
• MED-DSLC: Multi-Expert-Domain Classification via Domain Supervision and Logit Calibration [42.68764270775848]
  Vision-language models (VLMs) enable zero-shot classification by comparing image features with text prompts in a shared embedding space.<n>VLMs are often adapted to fine-grained domains using techniques such as LoRA.<n>This leads to a highly fragmented model ecosystem, with thousands of specialized models.
  arXiv  Detail & Related papers  (2026-07-13T01:18:32Z)
• Transferability for General Reasoning: An Automated Curriculum for Multi-Domain RLVR [59.527728403519745]
  Transfer-Aware Curriculum prioritizes domains whose updates broadly benefit the rest of the training suite.<n>It achieves the best macro-averaged accuracy on both Qwen3-1.7B and Llama3.2-3B.<n>It remains robust on imbalanced training mixtures where learnability-only curricula over-commit to dominant domains.
  arXiv  Detail & Related papers  (2026-06-23T21:10:29Z)
• How Useful is Causal Invariance for Domain Adaptation in Finite-Sample Settings? [58.740078141879984]
  Machine learning models often degrade when they are deployed on a target distribution that differs from the source distributions they were trained on.<n>Recent work in causality-based domain generalization has shown how shared causal structure can induce invariant predictors.<n>In this paper, we study when full or partial causal knowledge can provably improve supervised domain adaptation.
  arXiv  Detail & Related papers  (2026-06-10T21:07:49Z)
• Harmony in Diversity: Multi-domain Contrastive Policy Optimization for Large Reasoning Models [38.65301275836433]
  Post-training has significantly enhanced the reasoning capability of Large Reasoning Models (LRMs)<n> GRPO-style RL methods in multi-domain settings often fail to achieve consistent improvements across all domains due to inherent interference in policy optimization.<n>We propose Multi-domain Contrastive Policy Optimization (MCPO), which analyzes the structural relationships among rollouts and promotes cross-domain knowledge sharing.
  arXiv  Detail & Related papers  (2026-05-25T05:42:57Z)
• LLM-EDT: Large Language Model Enhanced Cross-domain Sequential Recommendation with Dual-phase Training [53.539682966282534]
  Cross-domain Sequential Recommendation (CDSR) has been proposed to enrich user-item interactions by incorporating information from various domains.<n>Despite current progress, the imbalance issue and transition issue hinder further development of CDSR.<n>We propose an LLMs Enhanced Cross-domain Sequential Recommendation with Dual-phase Training (LLM-EDT)
  arXiv  Detail & Related papers  (2025-11-25T05:18:04Z)
• WeaveRec: An LLM-Based Cross-Domain Sequential Recommendation Framework with Model Merging [24.949880939628386]
  We introduce WeaveRec, which cross-trains multiple LoRA modules with source and target domain data in a weaving fashion.<n>We provide a theoretical guarantee that WeaveRec can reduce the upper bound of the expected error in the target domain.
  arXiv  Detail & Related papers  (2025-10-30T14:37:15Z)
• Unsupervised Structural-Counterfactual Generation under Domain Shift [1.9652313502216219]
  We present a novel generative modeling challenge: generating counterfactual samples in a target domain based on factual observations from a source domain.<n>Our framework combines the posterior distribution of effect-intrinsic variables from the source domain with the prior distribution of domain-intrinsic variables from the target domain to synthesize the desired counterfactuals.
  arXiv  Detail & Related papers  (2025-02-17T16:48:16Z)
• Decoupled Training: Return of Frustratingly Easy Multi-Domain Learning [20.17925272562433]
  Multi-domain learning aims to train a model with minimal average risk across multiple overlapping but non-identical domains. We propose Decoupled Training (D-Train) as a frustratingly easy and hyper parameter-free multi-domain learning method. D-Train is a tri-phase general-to-specific training strategy that first pre-trains on all domains to warm up a root model, then post-trains on each domain by splitting into multi-heads, and finally fine-tunes the heads by fixing the backbone.
  arXiv  Detail & Related papers  (2023-09-19T04:06:41Z)
• Adversarial Bi-Regressor Network for Domain Adaptive Regression [52.5168835502987]
  It is essential to learn a cross-domain regressor to mitigate the domain shift. This paper proposes a novel method Adversarial Bi-Regressor Network (ABRNet) to seek more effective cross-domain regression model.
  arXiv  Detail & Related papers  (2022-09-20T18:38:28Z)
• Learning Invariant Representations and Risks for Semi-supervised Domain Adaptation [109.73983088432364]
  We propose the first method that aims to simultaneously learn invariant representations and risks under the setting of semi-supervised domain adaptation (Semi-DA) We introduce the LIRR algorithm for jointly textbfLearning textbfInvariant textbfRepresentations and textbfRisks.
  arXiv  Detail & Related papers  (2020-10-09T15:42:35Z)
• Deep Residual Correction Network for Partial Domain Adaptation [79.27753273651747]
  Deep domain adaptation methods have achieved appealing performance by learning transferable representations from a well-labeled source domain to a different but related unlabeled target domain. This paper proposes an efficiently-implemented Deep Residual Correction Network (DRCN) Comprehensive experiments on partial, traditional and fine-grained cross-domain visual recognition demonstrate that DRCN is superior to the competitive deep domain adaptation approaches.
  arXiv  Detail & Related papers  (2020-04-10T06:07:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.