CARE-RL: Capability-Aware Reinforcement Learning for Mitigating Cross-Domain Conflicts

• URL: http://arxiv.org/abs/2606.00609v1
• Date: Sat, 30 May 2026 08:18:40 GMT
• Title: CARE-RL: Capability-Aware Reinforcement Learning for Mitigating Cross-Domain Conflicts
• Abstract summary: We propose CARE-RL to combine protocol-aware reward generation with capability-aware optimization for mitigating cross-domain conflicts.<n>For non-verifiable tasks, the Protocol-Aware Generative Reward Model (PA-GRM) constructs prompt-level evaluation protocols and schemas before producing trace-conditioned rewards.<n>Experiments across math, chat, and instruction-following benchmarks show that CARE-RL consistently outperforms standard multi-domain RL baselines.
• Score: 9.061334667823193
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Reinforcement learning (RL) with verifiable rewards has achieved strong progress in reasoning-oriented LLMs, but extending it to multi-domain RL remains challenging due to reward unreliability in non-verifiable tasks and capability interference across domains. We propose CARE-RL to combine protocol-aware reward generation with capability-aware optimization for mitigating cross-domain conflicts. For non-verifiable tasks, the Protocol-Aware Generative Reward Model (PA-GRM) constructs prompt-level evaluation protocols and schemas before producing trace-conditioned rewards, enabling task-adaptive yet comparable evaluation of open-ended responses. For multi-domain optimization, Direction-Aware Capability Subspace Projection (DACSP) extracts historical capability directions from previous RL stages and modulates later updates by amplifying aligned components, suppressing conflicting components, and preserving orthogonal updates. Experiments across math, chat, and instruction-following benchmarks show that CARE-RL consistently outperforms standard multi-domain RL baselines, achieving Total Avg scores of 47.9 and 50.7 on Qwen2.5-7B and Qwen3-4B, respectively.

Related papers

• Learning to Act While Waiting: RL Finetuning of Generalist Robot Policies Under Inference Latency [38.99457938189648]
  We introduce a latency-aware framework, Asynchronous RL with Intermediate Information (ARLI), that enables RL-based improvement of generalist policies under inference delays.<n>Our framework builds on asynchronous inference approaches, which interleave action generation with execution to hide latency, and addresses its incompatibility with RL.<n>We evaluate our approach across simulated and real-world manipulation tasks, and find that it enables effective finetuning under inference delays where standard RL fails entirely.
  arXiv  Detail & Related papers  (2026-08-24T21:19:50Z)
• Co-RL: Unsupervised Reasoning Emerges from Diverse Cohort in Multi-agent RL [55.05743310621117]
  Co-RL is a framework in which multiple decoupled models, sharing no parameters, are simultaneously optimized through RL using rewards derived from their peers.<n>We show that increasing cohort diversity, through heterogeneous model families, sizes, and rephrased training samples, reduces the correlated errors that drive self-reinforcing feedback loops.
  arXiv  Detail & Related papers  (2026-08-18T01:16:02Z)
• Benchmarking the Limits of In-Context Reinforcement Learning for Ad-Hoc Teamwork [45.63941874462679]
  In-Context Reinforcement Learning (ICRL) has enabled foundation agents to adapt instantaneously to novel tasks, yet its efficacy in Ad-Hoc Teamwork (AHT)-where coordination with unknown partners is required-remains unexplored.<n>We introduce a large-scale benchmark ICRL4AHT, built upon a high- throughput JAX implementation of Overcooked-V2.<n>We evaluate representative history-conditioned ICRL algorithms, including Algorithm Distillation (AD) and Decision-Pretrained Transformer (DPT) across millions of transitions.
  arXiv  Detail & Related papers  (2026-05-23T06:39:21Z)
• Relative Score Policy Optimization for Diffusion Language Models [29.344961499429257]
  Diffusion large language models (dLLMs) offer a promising route to parallel and efficient text generation.<n>The lack of tractable sequence-level log-ratios forces existing methods to rely on high-variance ELBO-based approximations.<n>We propose textbfRelative textbfScore textbfPolicy textbfOptimization (RSPO) to calibrate noisy likelihood estimates in dLLMs.
  arXiv  Detail & Related papers  (2026-05-11T08:58:40Z)
• Controllable Exploration in Hybrid-Policy RLVR for Multi-Modal Reasoning [88.42566960813438]
  CalibRL is a hybrid-policy RLVR framework that supports controllable exploration with expert guidance.<n>CalibRL increases policy entropy in a guided manner and clarifies the target distribution.<n>Experiments across eight benchmarks, including both in-domain and out-of-domain settings, demonstrate consistent improvements.
  arXiv  Detail & Related papers  (2026-02-22T07:23:36Z)
• ProRAG: Process-Supervised Reinforcement Learning for Retrieval-Augmented Generation [54.071574153853994]
  ProRAG is a process-supervised reinforcement learning framework designed to integrate learned step-level supervision into the online optimization loop.<n>Our framework consists of four stages: (1) Supervised Policy Warmup to initialize the model with a structured reasoning format; (2) construction of an MCTS-based Process Reward Model (PRM) to quantify intermediate reasoning quality; (3) PRM-Guided Reasoning Refinement to align the policy with fine-grained process preferences; and (4) Process-Supervised Reinforcement Learning with a dual-granularity advantage mechanism.
  arXiv  Detail & Related papers  (2026-01-29T16:04:59Z)
• DARL: Encouraging Diverse Answers for General Reasoning without Verifiers [41.35516261603945]
  We propose DARL, a reinforcement learning framework that encourages the generation of diverse answers within a controlled deviation range from the reference.<n>Our framework is fully compatible with existing general reinforcement learning methods and can be seamlessly integrated without additional verifiers.
  arXiv  Detail & Related papers  (2026-01-21T06:23:55Z)
• ArenaRL: Scaling RL for Open-Ended Agents via Tournament-based Relative Ranking [84.07076200941474]
  ArenaRL is a reinforcement learning paradigm that shifts from pointwise scalar scoring to intra-group relative ranking.<n>We construct an intra-group adversarial arena and devise a tournament-based ranking scheme to obtain stable advantage signals.<n>Experiments show that ArenaRL substantially outperforms standard RL baselines.
  arXiv  Detail & Related papers  (2026-01-10T08:43:07Z)
• From Supervision to Exploration: What Does Protein Language Model Learn During Reinforcement Learning? [76.288870982181]
  Protein language models (PLMs) have advanced computational protein science through large-scale pretraining and scalable architectures.<n> reinforcement learning (RL) has broadened exploration and enabled precise multi-objective optimization in protein design.<n>We ask if RL improves sampling efficiency and, more importantly, if it reveals capabilities not captured by supervised learning.
  arXiv  Detail & Related papers  (2025-10-02T01:31:10Z)
• Agentic Reinforcement Learning with Implicit Step Rewards [92.26560379363492]
  Large language models (LLMs) are increasingly developed as autonomous agents using reinforcement learning (agentic RL)<n>We introduce implicit step rewards for agentic RL (iStar), a general credit-assignment strategy that integrates seamlessly with standard RL algorithms.<n>We evaluate our method on three challenging agent benchmarks, including WebShop and VisualSokoban, as well as open-ended social interactions with unverifiable rewards in SOTOPIA.
  arXiv  Detail & Related papers  (2025-09-23T16:15:42Z)
• RLPR: Extrapolating RLVR to General Domains without Verifiers [103.14103272635893]
  We propose RLPR, a simple verifier-free framework that extrapolates RLVR to broader general domains.<n>We find that addressing the high variance of this noisy probability reward is crucial to make it work.<n>RLPR consistently improves reasoning capabilities in both areas for Gemma, Llama, and Qwen based models.
  arXiv  Detail & Related papers  (2025-06-23T02:56:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.