Cascaded Transfer: Learning Many Tasks under Budget Constraints

• URL: http://arxiv.org/abs/2601.21513v1
• Date: Thu, 29 Jan 2026 10:28:08 GMT
• Title: Cascaded Transfer: Learning Many Tasks under Budget Constraints
• Authors: Eloi Campagne, Yvenn Amara-Ouali, Yannig Goude, Mathilde Mougeot, Argyris Kalogeratos,
• Abstract summary: Many-Task Learning refers to the setting where a large number of related tasks need to be learned, the exact relationships between tasks are not known.<n>We introduce the Cascaded Transfer Learning, a novel many-task transfer learning paradigm where information cascades hierarchically through tasks that are learned by individual models of the same class, while respecting given budget constraints.
• Score: 2.352067921771369
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Many-Task Learning refers to the setting where a large number of related tasks need to be learned, the exact relationships between tasks are not known. We introduce the Cascaded Transfer Learning, a novel many-task transfer learning paradigm where information (e.g. model parameters) cascades hierarchically through tasks that are learned by individual models of the same class, while respecting given budget constraints. The cascade is organized as a rooted tree that specifies the order in which tasks are learned and refined. We design a cascaded transfer mechanism deployed over a minimum spanning tree structure that connects the tasks according to a suitable distance measure, and allocates the available training budget along its branches. Experiments on synthetic and real many-task settings show that the resulting method enables more accurate and cost effective adaptation across large task collections compared to alternative approaches.

Related papers

• A Tensor Low-Rank Approximation for Value Functions in Multi-Task Reinforcement Learning [10.359616364592073]
  In pursuit of reinforcement learning systems that could train in physical environments, we investigate multi-task approaches.<n>A low-rank structure enforces the notion of similarity, without the need to explicitly prescribe which tasks are similar.<n>The efficiency of our low-rank tensor approach to multi-task learning is demonstrated in two numerical experiments.
  arXiv  Detail & Related papers  (2025-01-17T20:07:11Z)
• Transfer Learning for Structured Pruning under Limited Task Data [15.946734013984184]
  We propose a framework which combines structured pruning with transfer learning to reduce the need for task-specific data. We demonstrate that our framework results in pruned models with improved generalization over strong baselines.
  arXiv  Detail & Related papers  (2023-11-10T20:23:35Z)
• Fast Inference and Transfer of Compositional Task Structures for Few-shot Task Generalization [101.72755769194677]
  We formulate it as a few-shot reinforcement learning problem where a task is characterized by a subtask graph. Our multi-task subtask graph inferencer (MTSGI) first infers the common high-level task structure in terms of the subtask graph from the training tasks. Our experiment results on 2D grid-world and complex web navigation domains show that the proposed method can learn and leverage the common underlying structure of the tasks for faster adaptation to the unseen tasks.
  arXiv  Detail & Related papers  (2022-05-25T10:44:25Z)
• Sparsely Activated Mixture-of-Experts are Robust Multi-Task Learners [67.5865966762559]
  We study whether sparsely activated Mixture-of-Experts (MoE) improve multi-task learning. We devise task-aware gating functions to route examples from different tasks to specialized experts. This results in a sparsely activated multi-task model with a large number of parameters, but with the same computational cost as that of a dense model.
  arXiv  Detail & Related papers  (2022-04-16T00:56:12Z)
• Task Adaptive Parameter Sharing for Multi-Task Learning [114.80350786535952]
  Adaptive Task Adapting Sharing (TAPS) is a method for tuning a base model to a new task by adaptively modifying a small, task-specific subset of layers. Compared to other methods, TAPS retains high accuracy on downstream tasks while introducing few task-specific parameters. We evaluate our method on a suite of fine-tuning tasks and architectures (ResNet, DenseNet, ViT) and show that it achieves state-of-the-art performance while being simple to implement.
  arXiv  Detail & Related papers  (2022-03-30T23:16:07Z)
• Multi-Task Learning with Sequence-Conditioned Transporter Networks [67.57293592529517]
  We aim to solve multi-task learning through the lens of sequence-conditioning and weighted sampling. We propose a new suite of benchmark aimed at compositional tasks, MultiRavens, which allows defining custom task combinations. Second, we propose a vision-based end-to-end system architecture, Sequence-Conditioned Transporter Networks, which augments Goal-Conditioned Transporter Networks with sequence-conditioning and weighted sampling.
  arXiv  Detail & Related papers  (2021-09-15T21:19:11Z)
• Learning to Branch for Multi-Task Learning [12.49373126819798]
  We present an automated multi-task learning algorithm that learns where to share or branch within a network. We propose a novel tree-structured design space that casts a tree branching operation as a gumbel-softmax sampling procedure.
  arXiv  Detail & Related papers  (2020-06-02T19:23:21Z)
• Relatedness Measures to Aid the Transfer of Building Blocks among Multiple Tasks [3.0538120180981294]
  Multitask Learning is a learning paradigm that deals with multiple different tasks in parallel and transfers knowledge among them. XOF, a Learning System using tree-based programs to encode building blocks (metafeatures), constructs and collects features with rich discriminative information for classification tasks in an observed list. We propose a multiple-XOF system, called mXOF, that can dynamically adapt feature transfer among XOFs.
  arXiv  Detail & Related papers  (2020-05-08T10:26:59Z)
• Adversarial Continual Learning [99.56738010842301]
  We propose a hybrid continual learning framework that learns a disjoint representation for task-invariant and task-specific features. Our model combines architecture growth to prevent forgetting of task-specific skills and an experience replay approach to preserve shared skills.
  arXiv  Detail & Related papers  (2020-03-21T02:08:17Z)
• Hierarchical Reinforcement Learning as a Model of Human Task Interleaving [60.95424607008241]
  We develop a hierarchical model of supervisory control driven by reinforcement learning. The model reproduces known empirical effects of task interleaving. The results support hierarchical RL as a plausible model of task interleaving.
  arXiv  Detail & Related papers  (2020-01-04T17:53:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.