Transferability for General Reasoning: An Automated Curriculum for Multi-Domain RLVR

• URL: http://arxiv.org/abs/2606.25178v2
• Date: Sat, 27 Jun 2026 02:34:55 GMT
• Title: Transferability for General Reasoning: An Automated Curriculum for Multi-Domain RLVR
• Abstract summary: Transfer-Aware Curriculum prioritizes domains whose updates broadly benefit the rest of the training suite.<n>It achieves the best macro-averaged accuracy on both Qwen3-1.7B and Llama3.2-3B.<n>It remains robust on imbalanced training mixtures where learnability-only curricula over-commit to dominant domains.
• Score: 59.527728403519745
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Reinforcement learning with verifiable rewards (RLVR) has been extended from single-domain training to multi-domain reasoning suites spanning mathematics, programming, and science. However, the training curriculum (how often each domain is sampled) is typically fixed or hand-tuned, even though reasoning skills transfer unevenly across domains. Existing learnability-based curricula adapt to where the policy is currently improving, but are blind to whether a gradient step on the selected domain benefits the remaining domains. In this paper, we propose Transfer-Aware Curriculum (TAC), a bandit-style online curriculum that prioritizes domains whose updates broadly benefit the rest of the training suite. TAC repurposes signals already produced by RL training: per-domain advantages capture local learnability, and projected gradients, taken from the GRPO step being computed, estimate cross-domain transferability via gradient-geometry alignment, at negligible cost (<1% wall-clock overhead). Across a six-domain reasoning suite, TAC achieves the best macro-averaged accuracy on both Qwen3-1.7B and Llama3.2-3B, outperforming proportional random sampling, a hand-designed schedule, and a learnability-only bandit, and improving over the last of these by up to 2.8 points (10% relative). Ablations show performance degrades sharply when the transferability term is removed, and TAC remains robust on imbalanced training mixtures where learnability-only curricula over-commit to dominant domains. Our findings establish cross-domain transferability as a key signal for curriculum design in multi-domain RLVR.

Related papers

• Target-Aligned Bellman Backup for Cross-domain Offline Reinforcement Learning [7.282517489507499]
  Cross-domain offline reinforcement learning aims to improve policy learning in a target domain by leveraging data collected from a source domain.<n>Existing works typically assess the transferability of source-domain data by measuring its similarity to target-domain transitions, and implicitly perform transition-level selection.<n>We propose a method termed Target-Aligned Bellman Backup (TABB), which selectively leverages source-domain data by measuring their contribution to accurate Bellman target estimation in the target domain.
  arXiv  Detail & Related papers  (2026-05-21T12:09:11Z)
• Reinforced Curriculum Pre-Alignment for Domain-Adaptive VLMs [21.190823331753464]
  Vision-Language Models (VLMs) demonstrate remarkable general-purpose capabilities but often fall short in specialized domains.<n>We propose Reinforced Curriculum Pre-Alignment (RCPA), a novel post-training paradigm that introduces a curriculum-aware progressive modulation mechanism.
  arXiv  Detail & Related papers  (2026-02-11T11:04:37Z)
• Contrastive Adversarial Training for Unsupervised Domain Adaptation [2.432037584128226]
  Domain adversarial training has been successfully adopted for various domain adaptation tasks. Large models make adversarial training being easily biased towards source domain and hardly adapted to target domain. We propose contrastive adversarial training (CAT) approach that leverages the labeled source domain samples to reinforce and regulate the feature generation for target domain.
  arXiv  Detail & Related papers  (2024-07-17T17:59:21Z)
• FusDom: Combining In-Domain and Out-of-Domain Knowledge for Continuous Self-Supervised Learning [54.9235160379917]
  FusDom is a simple and novel methodology for SSL-based continued pre-training. FusDom learns speech representations that are robust and adaptive yet not forgetful of concepts seen in the past.
  arXiv  Detail & Related papers  (2023-12-20T13:50:05Z)
• Forget Less, Count Better: A Domain-Incremental Self-Distillation Learning Benchmark for Lifelong Crowd Counting [51.44987756859706]
  Off-the-shelf methods have some drawbacks to handle multiple domains. Lifelong Crowd Counting aims at alleviating the catastrophic forgetting and improving the generalization ability.
  arXiv  Detail & Related papers  (2022-05-06T15:37:56Z)
• Transferrable Contrastive Learning for Visual Domain Adaptation [108.98041306507372]
  Transferrable Contrastive Learning (TCL) is a self-supervised learning paradigm tailored for domain adaptation. TCL penalizes cross-domain intra-class domain discrepancy between source and target through a clean and novel contrastive loss. The free lunch is, thanks to the incorporation of contrastive learning, TCL relies on a moving-averaged key encoder that naturally achieves a temporally ensembled version of pseudo labels for target data.
  arXiv  Detail & Related papers  (2021-12-14T16:23:01Z)
• Contrastive Learning and Self-Training for Unsupervised Domain Adaptation in Semantic Segmentation [71.77083272602525]
  UDA attempts to provide efficient knowledge transfer from a labeled source domain to an unlabeled target domain. We propose a contrastive learning approach that adapts category-wise centroids across domains. We extend our method with self-training, where we use a memory-efficient temporal ensemble to generate consistent and reliable pseudo-labels.
  arXiv  Detail & Related papers  (2021-05-05T11:55:53Z)
• Prototypical Cross-domain Self-supervised Learning for Few-shot Unsupervised Domain Adaptation [91.58443042554903]
  We propose an end-to-end Prototypical Cross-domain Self-Supervised Learning (PCS) framework for Few-shot Unsupervised Domain Adaptation (FUDA) PCS not only performs cross-domain low-level feature alignment, but it also encodes and aligns semantic structures in the shared embedding space across domains. Compared with state-of-the-art methods, PCS improves the mean classification accuracy over different domain pairs on FUDA by 10.5%, 3.5%, 9.0%, and 13.2% on Office, Office-Home, VisDA-2017, and DomainNet, respectively.
  arXiv  Detail & Related papers  (2021-03-31T02:07:42Z)
• Physically-Constrained Transfer Learning through Shared Abundance Space for Hyperspectral Image Classification [14.840925517957258]
  We propose a new transfer learning scheme to bridge the gap between the source and target domains. The proposed method is referred to as physically-constrained transfer learning through shared abundance space.
  arXiv  Detail & Related papers  (2020-08-19T17:41:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.