Out-of-distribution Generalization via Partial Feature Decorrelation
- URL: http://arxiv.org/abs/2007.15241v4
- Date: Wed, 23 Feb 2022 10:01:23 GMT
- Title: Out-of-distribution Generalization via Partial Feature Decorrelation
- Authors: Xin Guo, Zhengxu Yu, Chao Xiang, Zhongming Jin, Jianqiang Huang, Deng
Cai, Xiaofei He, Xian-Sheng Hua
- Abstract summary: We present a novel Partial Feature Decorrelation Learning (PFDL) algorithm, which jointly optimize a feature decomposition network and the target image classification model.
The experiments on real-world datasets demonstrate that our method can improve the backbone model's accuracy on OOD image classification datasets.
- Score: 72.96261704851683
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Most deep-learning-based image classification methods assume that all samples
are generated under an independent and identically distributed (IID) setting.
However, out-of-distribution (OOD) generalization is more common in practice,
which means an agnostic context distribution shift between training and testing
environments. To address this problem, we present a novel Partial Feature
Decorrelation Learning (PFDL) algorithm, which jointly optimizes a feature
decomposition network and the target image classification model. The feature
decomposition network decomposes feature embeddings into the independent and
the correlated parts such that the correlations between features will be
highlighted. Then, the correlated features help learn a stable feature
representation by decorrelating the highlighted correlations while optimizing
the image classification model. We verify the correlation modeling ability of
the feature decomposition network on a synthetic dataset. The experiments on
real-world datasets demonstrate that our method can improve the backbone
model's accuracy on OOD image classification datasets.
Related papers
- DIVE: Subgraph Disagreement for Graph Out-of-Distribution Generalization [44.291382840373]
This paper addresses the challenge of out-of-distribution generalization in graph machine learning.
Traditional graph learning algorithms falter in real-world scenarios where this assumption fails.
A principal factor contributing to this suboptimal performance is the inherent simplicity bias of neural networks.
arXiv Detail & Related papers (2024-08-08T12:08:55Z) - Integrated feature analysis for deep learning interpretation and class activation maps [1.5943223374606597]
We propose an integrated feature analysis method to look closer into the intermediate features extracted by deep learning models.
This integrated feature analysis could provide information on overfitting, confounders, outliers in datasets.
The method was evaluated by calculating the consistency between the CAMs average class activation levels and the logits of the model.
arXiv Detail & Related papers (2024-07-01T10:10:57Z) - Common-Sense Bias Discovery and Mitigation for Classification Tasks [16.8259488742528]
We propose a framework to extract feature clusters in a dataset based on image descriptions.
The analyzed features and correlations are human-interpretable, so we name the method Common-Sense Bias Discovery (CSBD)
Experiments show that our method discovers novel biases on multiple classification tasks for two benchmark image datasets.
arXiv Detail & Related papers (2024-01-24T03:56:07Z) - Decom--CAM: Tell Me What You See, In Details! Feature-Level Interpretation via Decomposition Class Activation Map [23.71680014689873]
Class Activation Map (CAM) is widely used to interpret deep model predictions by highlighting object location.
This paper proposes a new two-stage interpretability method called the Decomposition Class Activation Map (Decom-CAM)
Our experiments demonstrate that the proposed Decom-CAM outperforms current state-of-the-art methods significantly.
arXiv Detail & Related papers (2023-05-27T14:33:01Z) - Adaptive Local-Component-aware Graph Convolutional Network for One-shot
Skeleton-based Action Recognition [54.23513799338309]
We present an Adaptive Local-Component-aware Graph Convolutional Network for skeleton-based action recognition.
Our method provides a stronger representation than the global embedding and helps our model reach state-of-the-art.
arXiv Detail & Related papers (2022-09-21T02:33:07Z) - Deep Relational Metric Learning [84.95793654872399]
This paper presents a deep relational metric learning framework for image clustering and retrieval.
We learn an ensemble of features that characterizes an image from different aspects to model both interclass and intraclass distributions.
Experiments on the widely-used CUB-200-2011, Cars196, and Stanford Online Products datasets demonstrate that our framework improves existing deep metric learning methods and achieves very competitive results.
arXiv Detail & Related papers (2021-08-23T09:31:18Z) - Accuracy on the Line: On the Strong Correlation Between
Out-of-Distribution and In-Distribution Generalization [89.73665256847858]
We show that out-of-distribution performance is strongly correlated with in-distribution performance for a wide range of models and distribution shifts.
Specifically, we demonstrate strong correlations between in-distribution and out-of-distribution performance on variants of CIFAR-10 & ImageNet.
We also investigate cases where the correlation is weaker, for instance some synthetic distribution shifts from CIFAR-10-C and the tissue classification dataset Camelyon17-WILDS.
arXiv Detail & Related papers (2021-07-09T19:48:23Z) - Model Fusion with Kullback--Leibler Divergence [58.20269014662046]
We propose a method to fuse posterior distributions learned from heterogeneous datasets.
Our algorithm relies on a mean field assumption for both the fused model and the individual dataset posteriors.
arXiv Detail & Related papers (2020-07-13T03:27:45Z) - Hierarchical regularization networks for sparsification based learning
on noisy datasets [0.0]
hierarchy follows from approximation spaces identified at successively finer scales.
For promoting model generalization at each scale, we also introduce a novel, projection based penalty operator across multiple dimension.
Results show the performance of the approach as a data reduction and modeling strategy on both synthetic and real datasets.
arXiv Detail & Related papers (2020-06-09T18:32:24Z) - Dynamic Federated Learning [57.14673504239551]
Federated learning has emerged as an umbrella term for centralized coordination strategies in multi-agent environments.
We consider a federated learning model where at every iteration, a random subset of available agents perform local updates based on their data.
Under a non-stationary random walk model on the true minimizer for the aggregate optimization problem, we establish that the performance of the architecture is determined by three factors, namely, the data variability at each agent, the model variability across all agents, and a tracking term that is inversely proportional to the learning rate of the algorithm.
arXiv Detail & Related papers (2020-02-20T15:00:54Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.