Multitask Learning with Learned Task Relationships

• URL: http://arxiv.org/abs/2510.10570v1
• Date: Sun, 12 Oct 2025 12:42:19 GMT
• Title: Multitask Learning with Learned Task Relationships
• Authors: Zirui Wan, Stefan Vlaski,
• Abstract summary: We introduce an algorithmic framework that strikes a balance between these extremes.<n>By modeling task relationships through a Gaussian Markov Random Field with an unknown precision matrix, we develop a strategy that jointly learns both the task relationships and the local models.<n>Our theoretical analysis quantifies the quality of the learned relationship, and our numerical experiments demonstrate its practical effectiveness.
• Score: 20.627498972435742
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Classical consensus-based strategies for federated and decentralized learning are statistically suboptimal in the presence of heterogeneous local data or task distributions. As a result, in recent years, there has been growing interest in multitask or personalized strategies, which allow individual agents to benefit from one another in pursuing locally optimal models without enforcing consensus. Existing strategies require either precise prior knowledge of the underlying task relationships or are fully non-parametric and instead rely on meta-learning or proximal constructions. In this work, we introduce an algorithmic framework that strikes a balance between these extremes. By modeling task relationships through a Gaussian Markov Random Field with an unknown precision matrix, we develop a strategy that jointly learns both the task relationships and the local models, allowing agents to self-organize in a way consistent with their individual data distributions. Our theoretical analysis quantifies the quality of the learned relationship, and our numerical experiments demonstrate its practical effectiveness.

Related papers

• Multi-View Graph Convolution Network for Internal Talent Recommendation Based on Enterprise Emails [5.282071089128208]
  Internal talent recommendation is a critical strategy for organizational continuity.<n>We propose a novel framework that models two distinct dimensions of an employee's position fit from email data.<n> Experiments show that our proposed gating-based fusion model significantly outperforms other fusion strategies.
  arXiv  Detail & Related papers  (2025-08-28T00:11:24Z)
• Personalized and Resilient Distributed Learning Through Opinion Dynamics [1.1499574149885023]
  We address two practical challenges of distributed learning in multi-agent network systems, namely personalization and resilience.<n>Motivated by a conceptual affinity between these two requirements, we devise a distributed learning algorithm that combines distributed gradient descent and the Friedkin-Johnsen model of opinion dynamics.<n>We numerically showcase the effectiveness of our algorithm on synthetic and real-world distributed learning tasks, where it achieves high global accuracy both for personalized models and with malicious agents compared to standard strategies.
  arXiv  Detail & Related papers  (2025-05-20T08:39:16Z)
• Task Groupings Regularization: Data-Free Meta-Learning with Heterogeneous Pre-trained Models [83.02797560769285]
  Data-Free Meta-Learning (DFML) aims to derive knowledge from a collection of pre-trained models without accessing their original data.<n>Current methods often overlook the heterogeneity among pre-trained models, which leads to performance degradation due to task conflicts.
  arXiv  Detail & Related papers  (2024-05-26T13:11:55Z)
• ADEPT: Hierarchical Bayes Approach to Personalized Federated Unsupervised Learning [7.19348361223786]
  We develop algorithms based on optimization criteria inspired by a hierarchical Bayesian statistical framework.<n>We develop adaptive algorithms that discover the balance between using limited local data and collaborative information.<n>We evaluate our proposed algorithms using synthetic and real data, demonstrating the effective sample amplification for personalized tasks.
  arXiv  Detail & Related papers  (2024-02-19T20:53:27Z)
• Multi-Agent Imitation Learning with Copulas [102.27052968901894]
  Multi-agent imitation learning aims to train multiple agents to perform tasks from demonstrations by learning a mapping between observations and actions. In this paper, we propose to use copula, a powerful statistical tool for capturing dependence among random variables, to explicitly model the correlation and coordination in multi-agent systems. Our proposed model is able to separately learn marginals that capture the local behavioral patterns of each individual agent, as well as a copula function that solely and fully captures the dependence structure among agents.
  arXiv  Detail & Related papers  (2021-07-10T03:49:41Z)
• Distributed Bayesian Online Learning for Cooperative Manipulation [9.582645137247667]
  We propose a novel distributed learning framework for the exemplary task of cooperative manipulation using Bayesian principles. Using only local state information each agent obtains an estimate of the object dynamics and grasp kinematics. Each estimate of the object dynamics and grasp kinematics is accompanied by a measure of uncertainty, which allows to guarantee a bounded prediction error with high probability.
  arXiv  Detail & Related papers  (2021-04-09T13:03:09Z)
• Byzantine Resilient Distributed Multi-Task Learning [6.850757447639822]
  We show that distributed algorithms for learning relatedness among tasks are not resilient in the presence of Byzantine agents. We propose an approach for Byzantine resilient distributed multi-task learning.
  arXiv  Detail & Related papers  (2020-10-25T04:32:52Z)
• Dif-MAML: Decentralized Multi-Agent Meta-Learning [54.39661018886268]
  We propose a cooperative multi-agent meta-learning algorithm, referred to as MAML or Dif-MAML. We show that the proposed strategy allows a collection of agents to attain agreement at a linear rate and to converge to a stationary point of the aggregate MAML. Simulation results illustrate the theoretical findings and the superior performance relative to the traditional non-cooperative setting.
  arXiv  Detail & Related papers  (2020-10-06T16:51:09Z)
• Learning while Respecting Privacy and Robustness to Distributional Uncertainties and Adversarial Data [66.78671826743884]
  The distributionally robust optimization framework is considered for training a parametric model. The objective is to endow the trained model with robustness against adversarially manipulated input data. Proposed algorithms offer robustness with little overhead.
  arXiv  Detail & Related papers  (2020-07-07T18:25:25Z)
• Meta-Reinforcement Learning Robust to Distributional Shift via Model Identification and Experience Relabeling [126.69933134648541]
  We present a meta-reinforcement learning algorithm that is both efficient and extrapolates well when faced with out-of-distribution tasks at test time. Our method is based on a simple insight: we recognize that dynamics models can be adapted efficiently and consistently with off-policy data.
  arXiv  Detail & Related papers  (2020-06-12T13:34:46Z)
• Dynamic Federated Learning [57.14673504239551]
  Federated learning has emerged as an umbrella term for centralized coordination strategies in multi-agent environments. We consider a federated learning model where at every iteration, a random subset of available agents perform local updates based on their data. Under a non-stationary random walk model on the true minimizer for the aggregate optimization problem, we establish that the performance of the architecture is determined by three factors, namely, the data variability at each agent, the model variability across all agents, and a tracking term that is inversely proportional to the learning rate of the algorithm.
  arXiv  Detail & Related papers  (2020-02-20T15:00:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.