FedDM: Enhancing Communication Efficiency and Handling Data Heterogeneity in Federated Diffusion Models

• URL: http://arxiv.org/abs/2407.14730v1
• Date: Sat, 20 Jul 2024 02:54:41 GMT
• Title: FedDM: Enhancing Communication Efficiency and Handling Data Heterogeneity in Federated Diffusion Models
• Abstract summary: FedDM is a novel training framework designed for the federated training of diffusion models. We propose a suite of training algorithms that leverage the U-Net architecture as the backbone for our diffusion models. We evaluate our algorithms on FashionMNIST (28x28 resolution), CIFAR-10 (32x32 resolution), and CelebA (64x64 resolution) for DDPMs, and LSUN Church Outdoors (256x256 resolution) for LDMs.
• Score: 9.646249924704021
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We introduce FedDM, a novel training framework designed for the federated training of diffusion models. Our theoretical analysis establishes the convergence of diffusion models when trained in a federated setting, presenting the specific conditions under which this convergence is guaranteed. We propose a suite of training algorithms that leverage the U-Net architecture as the backbone for our diffusion models. These include a basic Federated Averaging variant, FedDM-vanilla, FedDM-prox to handle data heterogeneity among clients, and FedDM-quant, which incorporates a quantization module to reduce the model update size, thereby enhancing communication efficiency across the federated network. We evaluate our algorithms on FashionMNIST (28x28 resolution), CIFAR-10 (32x32 resolution), and CelebA (64x64 resolution) for DDPMs, as well as LSUN Church Outdoors (256x256 resolution) for LDMs, focusing exclusively on the imaging modality. Our evaluation results demonstrate that FedDM algorithms maintain high generation quality across image resolutions. At the same time, the use of quantized updates and proximal terms in the local training objective significantly enhances communication efficiency (up to 4x) and model convergence, particularly in non-IID data settings, at the cost of increased FID scores (up to 1.75x).

Related papers

• FedDSR: Federated Deep Supervision and Regularization Towards Autonomous Driving [32.600054594223096]
  Federated Deep Supervision and Regularization (FedDSR) is a paradigm that incorporates multi-access intermediate layer supervision and regularization within federated AD system.<n>FedDSR achieves up to 8.93% improvement in mIoU and 28.57% reduction in training rounds, compared to other Federated Learning baselines.
  arXiv  Detail & Related papers  (2025-12-07T06:23:59Z)
• FedSparQ: Adaptive Sparse Quantization with Error Feedback for Robust & Efficient Federated Learning [7.461859467262201]
  Federated Learning (FL) enables collaborative model training across decentralized clients.<n>FL suffers from significant communication overhead due to the frequent exchange of high-dimensional model updates over constrained networks.<n>We present FedSparQ, a lightweight compression framework that dynamically sparsifies the gradient of each client.
  arXiv  Detail & Related papers  (2025-11-05T12:38:08Z)
• FedPhD: Federated Pruning with Hierarchical Learning of Diffusion Models [5.5058010121503]
  Federated Learning (FL) trains models over distributed clients' data.<n> challenges such as high communication costs and data heterogeneity persist in training DMs.<n>We introduce FedPhD, designed to efficiently train DMs in FL environments.
  arXiv  Detail & Related papers  (2025-07-08T23:24:07Z)
• Efficient Federated Learning with Timely Update Dissemination [54.668309196009204]
  Federated Learning (FL) has emerged as a compelling methodology for the management of distributed data.<n>We propose an efficient FL approach that capitalizes on additional downlink bandwidth resources to ensure timely update dissemination.
  arXiv  Detail & Related papers  (2025-07-08T14:34:32Z)
• Variational Autoencoding Discrete Diffusion with Enhanced Dimensional Correlations Modeling [48.96034602889216]
  Variencoding Discrete Diffusion (VADD) is a novel framework that enhances discrete diffusion with latent variable modeling.<n>By introducing an auxiliary recognition model, VADD enables stable training via variational lower bounds and amortized inference over the training set.<n> Empirical results on 2D toy data, pixel-level image generation, and text generation demonstrate that VADD consistently outperforms MDM baselines.
  arXiv  Detail & Related papers  (2025-05-23T01:45:47Z)
• MMaDA: Multimodal Large Diffusion Language Models [61.13527224215318]
  We introduce MMaDA, a novel class of multimodal diffusion foundation models.<n>It is designed to achieve superior performance across diverse domains such as textual reasoning, multimodal understanding, and text-to-image generation.
  arXiv  Detail & Related papers  (2025-05-21T17:59:05Z)
• Masked Autoencoders Are Effective Tokenizers for Diffusion Models [56.08109308294133]
  MAETok is an autoencoder that learns semantically rich latent space while maintaining reconstruction fidelity. MaETok achieves significant practical improvements, enabling a gFID of 1.69 with 76x faster training and 31x higher inference throughput for 512x512 generation.
  arXiv  Detail & Related papers  (2025-02-05T18:42:04Z)
• Mitigating Embedding Collapse in Diffusion Models for Categorical Data [52.90687881724333]
  We introduce CATDM, a continuous diffusion framework within the embedding space that stabilizes training. Experiments on benchmarks show that CATDM mitigates embedding collapse, yielding superior results on FFHQ, LSUN Churches, and LSUN Bedrooms.
  arXiv  Detail & Related papers  (2024-10-18T09:12:33Z)
• Steering Masked Discrete Diffusion Models via Discrete Denoising Posterior Prediction [88.65168366064061]
  We introduce Discrete Denoising Posterior Prediction (DDPP), a novel framework that casts the task of steering pre-trained MDMs as a problem of probabilistic inference. Our framework leads to a family of three novel objectives that are all simulation-free, and thus scalable. We substantiate our designs via wet-lab validation, where we observe transient expression of reward-optimized protein sequences.
  arXiv  Detail & Related papers  (2024-10-10T17:18:30Z)
• Cross-conditioned Diffusion Model for Medical Image to Image Translation [22.020931436223204]
  We introduce a Cross-conditioned Diffusion Model (CDM) for medical image-to-image translation. First, we propose a Modality-specific Representation Model (MRM) to model the distribution of target modalities. Then, we design a Modality-decoupled Diffusion Network (MDN) to efficiently and effectively learn the distribution from MRM.
  arXiv  Detail & Related papers  (2024-09-13T02:48:56Z)
• Model Inversion Attacks Through Target-Specific Conditional Diffusion Models [54.69008212790426]
  Model inversion attacks (MIAs) aim to reconstruct private images from a target classifier's training set, thereby raising privacy concerns in AI applications. Previous GAN-based MIAs tend to suffer from inferior generative fidelity due to GAN's inherent flaws and biased optimization within latent space. We propose Diffusion-based Model Inversion (Diff-MI) attacks to alleviate these issues.
  arXiv  Detail & Related papers  (2024-07-16T06:38:49Z)
• Training Diffusion Models with Federated Learning [0.7373617024876727]
  Training of diffusion-based models for image generation is predominantly controlled by a select few Big Tech companies. We propose a federated diffusion model scheme that enables the independent and collaborative training of diffusion models without exposing local data.
  arXiv  Detail & Related papers  (2024-06-18T13:02:48Z)
• FedDiff: Diffusion Model Driven Federated Learning for Multi-Modal and Multi-Clients [32.59184269562571]
  We propose a multi-modal collaborative diffusion federated learning framework called FedDiff. Our framework establishes a dual-branch diffusion model feature extraction setup, where the two modal data are inputted into separate branches of the encoder. Considering the challenge of private and efficient communication between multiple clients, we embed the diffusion model into the federated learning communication structure.
  arXiv  Detail & Related papers  (2023-11-16T02:29:37Z)
• Over-the-Air Federated Learning and Optimization [52.5188988624998]
  We focus on Federated learning (FL) via edge-the-air computation (AirComp) We describe the convergence of AirComp-based FedAvg (AirFedAvg) algorithms under both convex and non- convex settings. For different types of local updates that can be transmitted by edge devices (i.e., model, gradient, model difference), we reveal that transmitting in AirFedAvg may cause an aggregation error. In addition, we consider more practical signal processing schemes to improve the communication efficiency and extend the convergence analysis to different forms of model aggregation error caused by these signal processing schemes.
  arXiv  Detail & Related papers  (2023-10-16T05:49:28Z)
• FedDM: Iterative Distribution Matching for Communication-Efficient Federated Learning [87.08902493524556]
  Federated learning(FL) has recently attracted increasing attention from academia and industry. We propose FedDM to build the global training objective from multiple local surrogate functions. In detail, we construct synthetic sets of data on each client to locally match the loss landscape from original data.
  arXiv  Detail & Related papers  (2022-07-20T04:55:18Z)
• Fed-LAMB: Layerwise and Dimensionwise Locally Adaptive Optimization Algorithm [24.42828071396353]
  In the emerging paradigm of federated learning (FL), large amount of clients, such as mobile devices, are used to train on their respective data. Due to the low bandwidth, decentralized optimization methods need to shift the computation burden from those clients to those servers. We present Fed-LAMB, a novel learning method based on a layerwise, deep neural networks.
  arXiv  Detail & Related papers  (2021-10-01T16:54:31Z)
• Influence Estimation and Maximization via Neural Mean-Field Dynamics [60.91291234832546]
  We propose a novel learning framework using neural mean-field (NMF) dynamics for inference and estimation problems. Our framework can simultaneously learn the structure of the diffusion network and the evolution of node infection probabilities.
  arXiv  Detail & Related papers  (2021-06-03T00:02:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.