Ensemble Prediction of Task Affinity for Efficient Multi-Task Learning

• URL: http://arxiv.org/abs/2602.18591v1
• Date: Fri, 20 Feb 2026 19:57:38 GMT
• Title: Ensemble Prediction of Task Affinity for Efficient Multi-Task Learning
• Authors: Afiya Ayman, Ayan Mukhopadhyay, Aron Laszka,
• Abstract summary: We propose ETAP (Ensemble Task Affinity Predictor), a scalable framework that integrates principled and data-driven estimators to predict MTL performance gains.<n>We demonstrate that ETAP improves MTL gain prediction and enables more effective task grouping, outperforming state-of-the-art baselines across diverse application domains.
• Score: 8.604749005323557
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: A fundamental problem in multi-task learning (MTL) is identifying groups of tasks that should be learned together. Since training MTL models for all possible combinations of tasks is prohibitively expensive for large task sets, a crucial component of efficient and effective task grouping is predicting whether a group of tasks would benefit from learning together, measured as per-task performance gain over single-task learning. In this paper, we propose ETAP (Ensemble Task Affinity Predictor), a scalable framework that integrates principled and data-driven estimators to predict MTL performance gains. First, we consider the gradient-based updates of shared parameters in an MTL model to measure the affinity between a pair of tasks as the similarity between the parameter updates based on these tasks. This linear estimator, which we call affinity score, naturally extends to estimating affinity within a group of tasks. Second, to refine these estimates, we train predictors that apply non-linear transformations and correct residual errors, capturing complex and non-linear task relationships. We train these predictors on a limited number of task groups for which we obtain ground-truth gain values via multi-task learning for each group. We demonstrate on benchmark datasets that ETAP improves MTL gain prediction and enables more effective task grouping, outperforming state-of-the-art baselines across diverse application domains.

Related papers

• Identifying Task Groupings for Multi-Task Learning Using Pointwise V-Usable Information [11.545160697026514]
  We propose a metric of task relatedness based on task difficulty measured by pointwise V-usable information (PVI)<n>We conduct experiments to evaluate the feasibility of this metric for task grouping on 15 NLP datasets in the general, biomedical, and clinical domains.
  arXiv  Detail & Related papers  (2024-10-16T17:49:45Z)
• Interpetable Target-Feature Aggregation for Multi-Task Learning based on Bias-Variance Analysis [53.38518232934096]
  Multi-task learning (MTL) is a powerful machine learning paradigm designed to leverage shared knowledge across tasks to improve generalization and performance. We propose an MTL approach at the intersection between task clustering and feature transformation based on a two-phase iterative aggregation of targets and features. In both phases, a key aspect is to preserve the interpretability of the reduced targets and features through the aggregation with the mean, which is motivated by applications to Earth science.
  arXiv  Detail & Related papers  (2024-06-12T08:30:16Z)
• Multi-task learning via robust regularized clustering with non-convex group penalties [0.0]
  Multi-task learning (MTL) aims to improve estimation performance by sharing common information among related tasks. Existing MTL methods based on this assumption often ignore outlier tasks. We propose a novel MTL method called MultiTask Regularized Clustering (MTLRRC)
  arXiv  Detail & Related papers  (2024-04-04T07:09:43Z)
• Towards Principled Task Grouping for Multi-Task Learning [12.757893623250252]
  Multi-task learning (MTL) aims to leverage shared information among tasks to improve learning efficiency and accuracy.<n> MTL often struggles to effectively manage positive and negative transfer between tasks, which can hinder performance improvements.<n>This paper introduces a principled approach to task grouping in MTL, advancing beyond existing methods by addressing key theoretical and practical limitations.
  arXiv  Detail & Related papers  (2024-02-23T13:51:20Z)
• Mitigating Task Interference in Multi-Task Learning via Explicit Task Routing with Non-Learnable Primitives [19.90788777476128]
  Multi-task learning (MTL) seeks to learn a single model to accomplish multiple tasks by leveraging shared information among the tasks. Existing MTL models have been known to suffer from negative interference among tasks. We propose ETR-NLP to mitigate task interference through a synergistic combination of non-learnable primitives and explicit task routing.
  arXiv  Detail & Related papers  (2023-08-03T22:34:16Z)
• "It's a Match!" -- A Benchmark of Task Affinity Scores for Joint Learning [74.14961250042629]
  Multi-Task Learning (MTL) promises attractive, characterizing the conditions of its success is still an open problem in Deep Learning. Estimateing task affinity for joint learning is a key endeavor. Recent work suggests that the training conditions themselves have a significant impact on the outcomes of MTL. Yet, the literature is lacking a benchmark to assess the effectiveness of tasks affinity estimation techniques.
  arXiv  Detail & Related papers  (2023-01-07T15:16:35Z)
• AdaTask: A Task-aware Adaptive Learning Rate Approach to Multi-task Learning [19.201899503691266]
  We measure the task dominance degree of a parameter by the total updates of each task on this parameter. We propose a Task-wise Adaptive learning rate approach, AdaTask, to separate the emphaccumulative gradients and hence the learning rate of each task. Experiments on computer vision and recommender system MTL datasets demonstrate that AdaTask significantly improves the performance of dominated tasks.
  arXiv  Detail & Related papers  (2022-11-28T04:24:38Z)
• Variational Multi-Task Learning with Gumbel-Softmax Priors [105.22406384964144]
  Multi-task learning aims to explore task relatedness to improve individual tasks. We propose variational multi-task learning (VMTL), a general probabilistic inference framework for learning multiple related tasks.
  arXiv  Detail & Related papers  (2021-11-09T18:49:45Z)
• Semi-supervised Multi-task Learning for Semantics and Depth [88.77716991603252]
  Multi-Task Learning (MTL) aims to enhance the model generalization by sharing representations between related tasks for better performance. We propose the Semi-supervised Multi-Task Learning (MTL) method to leverage the available supervisory signals from different datasets. We present a domain-aware discriminator structure with various alignment formulations to mitigate the domain discrepancy issue among datasets.
  arXiv  Detail & Related papers  (2021-10-14T07:43:39Z)
• Task-Feature Collaborative Learning with Application to Personalized Attribute Prediction [166.87111665908333]
  We propose a novel multi-task learning method called Task-Feature Collaborative Learning (TFCL) Specifically, we first propose a base model with a heterogeneous block-diagonal structure regularizer to leverage the collaborative grouping of features and tasks. As a practical extension, we extend the base model by allowing overlapping features and differentiating the hard tasks.
  arXiv  Detail & Related papers  (2020-04-29T02:32:04Z)
• Multi-Task Learning for Dense Prediction Tasks: A Survey [87.66280582034838]
  Multi-task learning (MTL) techniques have shown promising results w.r.t. performance, computations and/or memory footprint. We provide a well-rounded view on state-of-the-art deep learning approaches for MTL in computer vision.
  arXiv  Detail & Related papers  (2020-04-28T09:15:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.