Game-theoretic distributed learning of generative models for heterogeneous data collections

• URL: http://arxiv.org/abs/2511.01740v1
• Date: Mon, 03 Nov 2025 16:52:04 GMT
• Title: Game-theoretic distributed learning of generative models for heterogeneous data collections
• Authors: Dmitrij Schlesinger, Boris Flach,
• Abstract summary: Local models can be treated as black boxes'' with the ability to learn their parameters from data.<n>We formulate the learning of the local models as a cooperative game starting from the principles of game theory.
• Score: 2.1485350418225244
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: One of the main challenges in distributed learning arises from the difficulty of handling heterogeneous local models and data. In light of the recent success of generative models, we propose to meet this challenge by building on the idea of exchanging synthetic data instead of sharing model parameters. Local models can then be treated as ``black boxes'' with the ability to learn their parameters from data and to generate data according to these parameters. Moreover, if the local models admit semi-supervised learning, we can extend the approach by enabling local models on different probability spaces. This allows to handle heterogeneous data with different modalities. We formulate the learning of the local models as a cooperative game starting from the principles of game theory. We prove the existence of a unique Nash equilibrium for exponential family local models and show that the proposed learning approach converges to this equilibrium. We demonstrate the advantages of our approach on standard benchmark vision datasets for image classification and conditional generation.

Related papers

• DecHW: Heterogeneous Decentralized Federated Learning Exploiting Second-Order Information [1.5658704610960568]
  This paper tackles the data and model heterogeneity by explicitly addressing the parameter level varying evidential credence across local models.<n>A novel aggregation approach is introduced that captures these parameter variations in local models and performs robust aggregation of neighbourhood local updates.<n>In extensive experiments with computer vision tasks, the proposed approach shows strong generalizability of local models at reduced communication costs.
  arXiv  Detail & Related papers  (2026-01-16T11:21:42Z)
• U-aggregation: Unsupervised Aggregation of Multiple Learning Algorithms [4.871473117968554]
  We propose an unsupervised model aggregation method, U-aggregation, for enhanced and robust performance in new populations.<n>Unlike existing supervised model aggregation or super learner approaches, U-aggregation assumes no observed labels or outcomes in the target population.<n>We demonstrate its potential real-world application by using U-aggregation to enhance genetic risk prediction of complex traits.
  arXiv  Detail & Related papers  (2025-01-30T01:42:51Z)
• Task Groupings Regularization: Data-Free Meta-Learning with Heterogeneous Pre-trained Models [83.02797560769285]
  Data-Free Meta-Learning (DFML) aims to derive knowledge from a collection of pre-trained models without accessing their original data.<n>Current methods often overlook the heterogeneity among pre-trained models, which leads to performance degradation due to task conflicts.
  arXiv  Detail & Related papers  (2024-05-26T13:11:55Z)
• Towards Theoretical Understandings of Self-Consuming Generative Models [56.84592466204185]
  This paper tackles the emerging challenge of training generative models within a self-consuming loop. We construct a theoretical framework to rigorously evaluate how this training procedure impacts the data distributions learned by future models. We present results for kernel density estimation, delivering nuanced insights such as the impact of mixed data training on error propagation.
  arXiv  Detail & Related papers  (2024-02-19T02:08:09Z)
• Federated Learning with Projected Trajectory Regularization [65.6266768678291]
  Federated learning enables joint training of machine learning models from distributed clients without sharing their local data. One key challenge in federated learning is to handle non-identically distributed data across the clients. We propose a novel federated learning framework with projected trajectory regularization (FedPTR) for tackling the data issue.
  arXiv  Detail & Related papers  (2023-12-22T02:12:08Z)
• Federated Learning of Models Pre-Trained on Different Features with Consensus Graphs [19.130197923214123]
  Learning an effective global model on private and decentralized datasets has become an increasingly important challenge of machine learning. We propose a feature fusion approach that extracts local representations from local models and incorporates them into a global representation that improves the prediction performance. This paper presents solutions to these problems and demonstrates them in real-world applications on time series data such as power grids and traffic networks.
  arXiv  Detail & Related papers  (2023-06-02T02:24:27Z)
• Generative Adversarial Reduced Order Modelling [0.0]
  We present GAROM, a new approach for reduced order modelling (ROM) based on generative adversarial networks (GANs)<n>GANs have the potential to learn data distribution and generate more realistic data.<n>In this work, we combine the GAN and ROM framework, by introducing a data-driven generative adversarial model able to learn solutions to parametric differential equations.
  arXiv  Detail & Related papers  (2023-05-25T09:23:33Z)
• Improving Heterogeneous Model Reuse by Density Estimation [105.97036205113258]
  This paper studies multiparty learning, aiming to learn a model using the private data of different participants. Model reuse is a promising solution for multiparty learning, assuming that a local model has been trained for each party.
  arXiv  Detail & Related papers  (2023-05-23T09:46:54Z)
• Learning Data Representations with Joint Diffusion Models [20.25147743706431]
  Joint machine learning models that allow synthesizing and classifying data often offer uneven performance between those tasks or are unstable to train. We extend the vanilla diffusion model with a classifier that allows for stable joint end-to-end training with shared parameterization between those objectives. The resulting joint diffusion model outperforms recent state-of-the-art hybrid methods in terms of both classification and generation quality on all evaluated benchmarks.
  arXiv  Detail & Related papers  (2023-01-31T13:29:19Z)
• Dataless Knowledge Fusion by Merging Weights of Language Models [47.432215933099016]
  Fine-tuning pre-trained language models has become the prevalent paradigm for building downstream NLP models.<n>This creates a barrier to fusing knowledge across individual models to yield a better single model.<n>We propose a dataless knowledge fusion method that merges models in their parameter space.
  arXiv  Detail & Related papers  (2022-12-19T20:46:43Z)
• Synthetic Model Combination: An Instance-wise Approach to Unsupervised Ensemble Learning [92.89846887298852]
  Consider making a prediction over new test data without any opportunity to learn from a training set of labelled data. Give access to a set of expert models and their predictions alongside some limited information about the dataset used to train them.
  arXiv  Detail & Related papers  (2022-10-11T10:20:31Z)
• Towards Understanding and Mitigating Dimensional Collapse in Heterogeneous Federated Learning [112.69497636932955]
  Federated learning aims to train models across different clients without the sharing of data for privacy considerations. We study how data heterogeneity affects the representations of the globally aggregated models. We propose sc FedDecorr, a novel method that can effectively mitigate dimensional collapse in federated learning.
  arXiv  Detail & Related papers  (2022-10-01T09:04:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.