Treatment Learning Causal Transformer for Noisy Image Classification
- URL: http://arxiv.org/abs/2203.15529v2
- Date: Mon, 30 Oct 2023 06:22:50 GMT
- Title: Treatment Learning Causal Transformer for Noisy Image Classification
- Authors: Chao-Han Huck Yang, I-Te Danny Hung, Yi-Chieh Liu, Pin-Yu Chen
- Abstract summary: In this work, we incorporate this binary information of "existence of noise" as treatment into image classification tasks to improve prediction accuracy.
Motivated from causal variational inference, we propose a transformer-based architecture, that uses a latent generative model to estimate robust feature representations for noise image classification.
We also create new noisy image datasets incorporating a wide range of noise factors for performance benchmarking.
- Score: 62.639851972495094
- License: http://creativecommons.org/licenses/by-sa/4.0/
- Abstract: Current top-notch deep learning (DL) based vision models are primarily based
on exploring and exploiting the inherent correlations between training data
samples and their associated labels. However, a known practical challenge is
their degraded performance against "noisy" data, induced by different
circumstances such as spurious correlations, irrelevant contexts, domain shift,
and adversarial attacks. In this work, we incorporate this binary information
of "existence of noise" as treatment into image classification tasks to improve
prediction accuracy by jointly estimating their treatment effects. Motivated
from causal variational inference, we propose a transformer-based architecture,
Treatment Learning Causal Transformer (TLT), that uses a latent generative
model to estimate robust feature representations from current observational
input for noise image classification. Depending on the estimated noise level
(modeled as a binary treatment factor), TLT assigns the corresponding inference
network trained by the designed causal loss for prediction. We also create new
noisy image datasets incorporating a wide range of noise factors (e.g., object
masking, style transfer, and adversarial perturbation) for performance
benchmarking. The superior performance of TLT in noisy image classification is
further validated by several refutation evaluation metrics. As a by-product,
TLT also improves visual salience methods for perceiving noisy images.
Related papers
- DN-CL: Deep Symbolic Regression against Noise via Contrastive Learning [12.660401635672969]
We propose textittextbfDeep Regression against textbfNoise via textbfContrastive textbfL earning (DN-CL).
DN-CL employs two parameter-sharing encoders to embed data points from various data transformations into feature shields against noise.
Our experiments indicate that DN-CL demonstrates superior performance in handling both noisy and clean data.
arXiv Detail & Related papers (2024-06-21T03:13:40Z) - Learning with Noisy Foundation Models [95.50968225050012]
This paper is the first work to comprehensively understand and analyze the nature of noise in pre-training datasets.
We propose a tuning method (NMTune) to affine the feature space to mitigate the malignant effect of noise and improve generalization.
arXiv Detail & Related papers (2024-03-11T16:22:41Z) - Transformer-based Clipped Contrastive Quantization Learning for
Unsupervised Image Retrieval [15.982022297570108]
Unsupervised image retrieval aims to learn the important visual characteristics without any given level to retrieve the similar images for a given query image.
In this paper, we propose a TransClippedCLR model by encoding the global context of an image using Transformer having local context through patch based processing.
Results using the proposed clipped contrastive learning are greatly improved on all datasets as compared to same backbone network with vanilla contrastive learning.
arXiv Detail & Related papers (2024-01-27T09:39:11Z) - Learning to Mask and Permute Visual Tokens for Vision Transformer
Pre-Training [59.923672191632065]
We propose a new self-supervised pre-training approach, named Masked and Permuted Vision Transformer (MaPeT)
MaPeT employs autoregressive and permuted predictions to capture intra-patch dependencies.
Our results demonstrate that MaPeT achieves competitive performance on ImageNet.
arXiv Detail & Related papers (2023-06-12T18:12:19Z) - Advancing Unsupervised Low-light Image Enhancement: Noise Estimation, Illumination Interpolation, and Self-Regulation [55.07472635587852]
Low-Light Image Enhancement (LLIE) techniques have made notable advancements in preserving image details and enhancing contrast.
These approaches encounter persistent challenges in efficiently mitigating dynamic noise and accommodating diverse low-light scenarios.
We first propose a method for estimating the noise level in low light images in a quick and accurate way.
We then devise a Learnable Illumination Interpolator (LII) to satisfy general constraints between illumination and input.
arXiv Detail & Related papers (2023-05-17T13:56:48Z) - NLIP: Noise-robust Language-Image Pre-training [95.13287735264937]
We propose a principled Noise-robust Language-Image Pre-training framework (NLIP) to stabilize pre-training via two schemes: noise-harmonization and noise-completion.
Our NLIP can alleviate the common noise effects during image-text pre-training in a more efficient way.
arXiv Detail & Related papers (2022-12-14T08:19:30Z) - Embedding contrastive unsupervised features to cluster in- and
out-of-distribution noise in corrupted image datasets [18.19216557948184]
Using search engines for web image retrieval is a tempting alternative to manual curation when creating an image dataset.
Their main drawback remains the proportion of incorrect (noisy) samples retrieved.
We propose a two stage algorithm starting with a detection step where we use unsupervised contrastive feature learning.
We find that the alignment and uniformity principles of contrastive learning allow OOD samples to be linearly separated from ID samples on the unit hypersphere.
arXiv Detail & Related papers (2022-07-04T16:51:56Z) - Evaluating and Mitigating Bias in Image Classifiers: A Causal
Perspective Using Counterfactuals [27.539001365348906]
We present a method for generating counterfactuals by incorporating a structural causal model (SCM) in an improved variant of Adversarially Learned Inference (ALI)
We show how to explain a pre-trained machine learning classifier, evaluate its bias, and mitigate the bias using a counterfactual regularizer.
arXiv Detail & Related papers (2020-09-17T13:19:31Z) - Variational Denoising Network: Toward Blind Noise Modeling and Removal [59.36166491196973]
Blind image denoising is an important yet very challenging problem in computer vision.
We propose a new variational inference method, which integrates both noise estimation and image denoising.
arXiv Detail & Related papers (2019-08-29T15:54:06Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.