Exploiting Correlations in Federated Learning: Opportunities and Practical Limitations

• URL: http://arxiv.org/abs/2604.14751v1
• Date: Thu, 16 Apr 2026 08:03:54 GMT
• Title: Exploiting Correlations in Federated Learning: Opportunities and Practical Limitations
• Abstract summary: We classify gradient and model compression schemes into three categories based on the type of correlations they exploit.<n>We propose quantitative metrics for measuring their magnitude, and reinterpret existing compression methods through this unified correlation-based framework.<n>Motivated by these findings, we propose two adaptive compression designs that actively switch between different compression modes.
• Score: 9.703749869278269
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The communication bottleneck in federated learning (FL) has spurred extensive research into techniques to reduce the volume of data exchanged between client devices and the central parameter server. In this paper, we systematically classify gradient and model compression schemes into three categories based on the type of correlations they exploit: structural, temporal, and spatial. We examine the sources of such correlations, propose quantitative metrics for measuring their magnitude, and reinterpret existing compression methods through this unified correlation-based framework. Our experimental studies demonstrate that the degrees of structural, temporal, and spatial correlations vary significantly depending on task complexity, model architecture, and algorithmic configurations. These findings suggest that algorithm designers should carefully evaluate correlation assumptions under specific deployment scenarios rather than assuming that they are always present. Motivated by these findings, we propose two adaptive compression designs that actively switch between different compression modes based on the measured correlation strength, and we evaluate their performance gains relative to conventional non-adaptive approaches. In summary, our unified taxonomy provides a clean and principled foundation for developing more effective and application-specific compression techniques for FL systems.

Related papers

• FlexCausal: Flexible Causal Disentanglement via Structural Flow Priors and Manifold-Aware Interventions [1.7114074082429929]
  Causal Disentangled Representation Learning aims to learn and disentangle low dimensional representations from observations.<n>We propose FlexCausal, a novel CDRL framework based on a block-diagonal covariance VAE.<n>Our framework ensures a precise structural correspondence between the learned latent subspaces and the ground-truth causal relations.
  arXiv  Detail & Related papers  (2026-01-29T11:30:53Z)
• SCALAR: Self-Calibrating Adaptive Latent Attention Representation Learning [2.152549977410309]
  High-dimensional, heterogeneous data with complex feature interactions pose significant challenges for traditional predictive modeling approaches.<n>Our architecture introduces an adaptive kernel-based attention mechanism that processes distinct feature groups separately before integration.<n> Experimental results show substantial improvements in performance metrics, compared to the state-of-the-art methods across diverse datasets.
  arXiv  Detail & Related papers  (2025-10-18T12:41:18Z)
• ISCS: Parameter-Guided Channel Ordering and Grouping for Learned Image Compression [11.645256028223272]
  We propose a dataset-agnostic method to identify and organize important channels in pretrained LIC models.<n>Our method effectively reduces computation while maintaining reconstruction quality, providing a practical and modular enhancement to existing learned compression frameworks.
  arXiv  Detail & Related papers  (2025-09-21T00:44:15Z)
• Neural Network Reprogrammability: A Unified Theme on Model Reprogramming, Prompt Tuning, and Prompt Instruction [57.19302613163439]
  We introduce neural network reprogrammability as a unifying framework for model adaptation.<n>We present a taxonomy that categorizes such information manipulation approaches across four key dimensions.<n>We also analyze remaining technical challenges and ethical considerations.
  arXiv  Detail & Related papers  (2025-06-05T05:42:27Z)
• Enhancing CTR Prediction with De-correlated Expert Networks [45.50697497028273]
  We propose a De-Correlated MoE (D-MoE) framework, which introduces a Cross-Expert De-Correlation loss to minimize expert correlations.<n>We show that D-MoE achieves a significant 1.19% Gross Merchandise Volume (GMV) lift compared to the Multi-Embedding MoE baseline.
  arXiv  Detail & Related papers  (2025-05-23T14:04:38Z)
• Aggregation on Learnable Manifolds for Asynchronous Federated Optimization [3.8208848658169763]
  We introduce a geometric framework that casts aggregation as curve learning.<n>Within this, we propose AsyncBezier, which replaces linear aggregation with low-degree curvature components.<n>We show that these gains are preserved even when other methods are allocated a higher local compute budget.
  arXiv  Detail & Related papers  (2025-03-18T16:36:59Z)
• Aggregation Weighting of Federated Learning via Generalization Bound Estimation [65.8630966842025]
  Federated Learning (FL) typically aggregates client model parameters using a weighting approach determined by sample proportions. We replace the aforementioned weighting method with a new strategy that considers the generalization bounds of each local model.
  arXiv  Detail & Related papers  (2023-11-10T08:50:28Z)
• Order-based Structure Learning with Normalizing Flows [7.972479571606131]
  Estimating causal structure of observational data is a challenging search problem that scales super-exponentially with graph size. Existing methods use continuous relaxations to make this problem computationally tractable but often restrict the data-generating process to additive noise models (ANMs) We present Order-based Structure Learning with Normalizing Flows (OSLow), a framework that relaxes these assumptions using autoregressive normalizing flows.
  arXiv  Detail & Related papers  (2023-08-14T22:17:33Z)
• Less is More: Mitigate Spurious Correlations for Open-Domain Dialogue Response Generation Models by Causal Discovery [52.95935278819512]
  We conduct the first study on spurious correlations for open-domain response generation models based on a corpus CGDIALOG curated in our work. Inspired by causal discovery algorithms, we propose a novel model-agnostic method for training and inference of response generation model.
  arXiv  Detail & Related papers  (2023-03-02T06:33:48Z)
• Accuracy on the Line: On the Strong Correlation Between Out-of-Distribution and In-Distribution Generalization [89.73665256847858]
  We show that out-of-distribution performance is strongly correlated with in-distribution performance for a wide range of models and distribution shifts. Specifically, we demonstrate strong correlations between in-distribution and out-of-distribution performance on variants of CIFAR-10 & ImageNet. We also investigate cases where the correlation is weaker, for instance some synthetic distribution shifts from CIFAR-10-C and the tissue classification dataset Camelyon17-WILDS.
  arXiv  Detail & Related papers  (2021-07-09T19:48:23Z)
• Semantic Correspondence with Transformers [68.37049687360705]
  We propose Cost Aggregation with Transformers (CATs) to find dense correspondences between semantically similar images. We include appearance affinity modelling to disambiguate the initial correlation maps and multi-level aggregation. We conduct experiments to demonstrate the effectiveness of the proposed model over the latest methods and provide extensive ablation studies.
  arXiv  Detail & Related papers  (2021-06-04T14:39:03Z)
• Estimating the Effects of Continuous-valued Interventions using Generative Adversarial Networks [103.14809802212535]
  We build on the generative adversarial networks (GANs) framework to address the problem of estimating the effect of continuous-valued interventions. Our model, SCIGAN, is flexible and capable of simultaneously estimating counterfactual outcomes for several different continuous interventions. To address the challenges presented by shifting to continuous interventions, we propose a novel architecture for our discriminator.
  arXiv  Detail & Related papers  (2020-02-27T18:46:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.