Federated Deconfounding and Debiasing Learning for Out-of-Distribution Generalization

• URL: http://arxiv.org/abs/2505.04979v2
• Date: Sun, 11 May 2025 02:37:25 GMT
• Title: Federated Deconfounding and Debiasing Learning for Out-of-Distribution Generalization
• Authors: Zhuang Qi, Sijin Zhou, Lei Meng, Han Hu, Han Yu, Xiangxu Meng,
• Abstract summary: Attribute bias in federated learning (FL) typically leads local models to optimize inconsistently due to the learning of non-causal associations.<n>We propose the underlineFederated underlineDeconfounding and underlineDebiasing underlineLearning (FedDDL) method.<n>It constructs a structured causal graph to analyze the model inference process, and performs backdoor adjustment to eliminate confounding paths.
• Score: 25.181305392387802
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Attribute bias in federated learning (FL) typically leads local models to optimize inconsistently due to the learning of non-causal associations, resulting degraded performance. Existing methods either use data augmentation for increasing sample diversity or knowledge distillation for learning invariant representations to address this problem. However, they lack a comprehensive analysis of the inference paths, and the interference from confounding factors limits their performance. To address these limitations, we propose the \underline{Fed}erated \underline{D}econfounding and \underline{D}ebiasing \underline{L}earning (FedDDL) method. It constructs a structured causal graph to analyze the model inference process, and performs backdoor adjustment to eliminate confounding paths. Specifically, we design an intra-client deconfounding learning module for computer vision tasks to decouple background and objects, generating counterfactual samples that establish a connection between the background and any label, which stops the model from using the background to infer the label. Moreover, we design an inter-client debiasing learning module to construct causal prototypes to reduce the proportion of the background in prototype components. Notably, it bridges the gap between heterogeneous representations via causal prototypical regularization. Extensive experiments on 2 benchmarking datasets demonstrate that \methodname{} significantly enhances the model capability to focus on main objects in unseen data, leading to 4.5\% higher Top-1 Accuracy on average over 9 state-of-the-art existing methods.

Related papers

• Empowering Vision Transformers with Multi-Scale Causal Intervention for Long-Tailed Image Classification [12.122203089278738]
  Causal inference has emerged as a promising approach to mitigate long-tail classification by handling the biases introduced by class imbalance.<n>This paper investigates the influence of existing causal models on CNNs and ViT variants.<n>It proposes TSCNet, a two-stage causal modeling method to discover fine-grained causal associations.
  arXiv  Detail & Related papers  (2025-05-13T02:23:55Z)
• Global Intervention and Distillation for Federated Out-of-Distribution Generalization [7.905159090314987]
  Attribute skew in federated learning leads local models to focus on learning non-causal associations.<n>This paper presents FedGID, which utilizes diverse attribute features for backdoor adjustment to break the spurious association between background and label.<n> Experimental results on three datasets demonstrate that FedGID enhances the model's ability to focus on the main subjects in unseen data.
  arXiv  Detail & Related papers  (2025-04-01T14:36:24Z)
• FedSA: A Unified Representation Learning via Semantic Anchors for Prototype-based Federated Learning [4.244188591221394]
  We propose a novel framework named Federated Learning via Semantic Anchors (FedSA) to decouple the generation of prototypes from local representation learning.<n>FedSA significantly outperforms existing prototype-based FL methods on various classification tasks.
  arXiv  Detail & Related papers  (2025-01-09T16:10:03Z)
• A Plug-and-Play Method for Rare Human-Object Interactions Detection by Bridging Domain Gap [50.079224604394]
  We present a novel model-agnostic framework called textbfContext-textbfEnhanced textbfFeature textbfAment (CEFA) CEFA consists of a feature alignment module and a context enhancement module. Our method can serve as a plug-and-play module to improve the detection performance of HOI models on rare categories.
  arXiv  Detail & Related papers  (2024-07-31T08:42:48Z)
• Decoupled Prototype Learning for Reliable Test-Time Adaptation [50.779896759106784]
  Test-time adaptation (TTA) is a task that continually adapts a pre-trained source model to the target domain during inference. One popular approach involves fine-tuning model with cross-entropy loss according to estimated pseudo-labels. This study reveals that minimizing the classification error of each sample causes the cross-entropy loss's vulnerability to label noise. We propose a novel Decoupled Prototype Learning (DPL) method that features prototype-centric loss computation.
  arXiv  Detail & Related papers  (2024-01-15T03:33:39Z)
• TDCGL: Two-Level Debiased Contrastive Graph Learning for Recommendation [1.5836776102398225]
  Long-tailed distribution of entities of KG and noise issues in the real world make item-entity dependent relations deviate from reflecting true characteristics. We design the Two-Level Debiased Contrastive Learning (TDCL) and deploy it in the knowledge graph. Considerable experiments on open-source datasets demonstrate that our method has excellent anti-noise capability.
  arXiv  Detail & Related papers  (2023-10-01T03:56:38Z)
• Universal Domain Adaptation from Foundation Models: A Baseline Study [58.51162198585434]
  We make empirical studies of state-of-the-art UniDA methods using foundation models. We introduce textitCLIP distillation, a parameter-free method specifically designed to distill target knowledge from CLIP models. Although simple, our method outperforms previous approaches in most benchmark tasks.
  arXiv  Detail & Related papers  (2023-05-18T16:28:29Z)
• Towards Understanding and Mitigating Dimensional Collapse in Heterogeneous Federated Learning [112.69497636932955]
  Federated learning aims to train models across different clients without the sharing of data for privacy considerations. We study how data heterogeneity affects the representations of the globally aggregated models. We propose sc FedDecorr, a novel method that can effectively mitigate dimensional collapse in federated learning.
  arXiv  Detail & Related papers  (2022-10-01T09:04:17Z)
• DRFLM: Distributionally Robust Federated Learning with Inter-client Noise via Local Mixup [58.894901088797376]
  federated learning has emerged as a promising approach for training a global model using data from multiple organizations without leaking their raw data. We propose a general framework to solve the above two challenges simultaneously. We provide comprehensive theoretical analysis including robustness analysis, convergence analysis, and generalization ability.
  arXiv  Detail & Related papers  (2022-04-16T08:08:29Z)
• Learning Bias-Invariant Representation by Cross-Sample Mutual Information Minimization [77.8735802150511]
  We propose a cross-sample adversarial debiasing (CSAD) method to remove the bias information misused by the target task. The correlation measurement plays a critical role in adversarial debiasing and is conducted by a cross-sample neural mutual information estimator. We conduct thorough experiments on publicly available datasets to validate the advantages of the proposed method over state-of-the-art approaches.
  arXiv  Detail & Related papers  (2021-08-11T21:17:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.