Learning Causal Transition Matrix for Instance-dependent Label Noise

• URL: http://arxiv.org/abs/2412.13516v2
• Date: Tue, 07 Jan 2025 03:08:39 GMT
• Title: Learning Causal Transition Matrix for Instance-dependent Label Noise
• Authors: Jiahui Li, Tai-Wei Chang, Kun Kuang, Ximing Li, Long Chen, Jun Zhou,
• Abstract summary: We study the data generation process of noisy labels from a causal perspective. An unobservable latent variable can affect either the instance itself, the label annotation procedure, or both. We have designed a novel training framework that explicitly models this causal relationship.
• Score: 40.634344530749324
• License:
• Abstract: Noisy labels are both inevitable and problematic in machine learning methods, as they negatively impact models' generalization ability by causing overfitting. In the context of learning with noise, the transition matrix plays a crucial role in the design of statistically consistent algorithms. However, the transition matrix is often considered unidentifiable. One strand of methods typically addresses this problem by assuming that the transition matrix is instance-independent; that is, the probability of mislabeling a particular instance is not influenced by its characteristics or attributes. This assumption is clearly invalid in complex real-world scenarios. To better understand the transition relationship and relax this assumption, we propose to study the data generation process of noisy labels from a causal perspective. We discover that an unobservable latent variable can affect either the instance itself, the label annotation procedure, or both, which complicates the identification of the transition matrix. To address various scenarios, we have unified these observations within a new causal graph. In this graph, the input instance is divided into a noise-resistant component and a noise-sensitive component based on whether they are affected by the latent variable. These two components contribute to identifying the ``causal transition matrix'', which approximates the true transition matrix with theoretical guarantee. In line with this, we have designed a novel training framework that explicitly models this causal relationship and, as a result, achieves a more accurate model for inferring the clean label.

Related papers

• Multi-Label Noise Transition Matrix Estimation with Label Correlations: Theory and Algorithm [73.94839250910977]
  Noisy multi-label learning has garnered increasing attention due to the challenges posed by collecting large-scale accurate labels. The introduction of transition matrices can help model multi-label noise and enable the development of statistically consistent algorithms. We propose a novel estimator that leverages label correlations without the need for anchor points or precise fitting of noisy class posteriors.
  arXiv  Detail & Related papers  (2023-09-22T08:35:38Z)
• Instance-Dependent Label-Noise Learning with Manifold-Regularized Transition Matrix Estimation [172.81824511381984]
  The transition matrix T(x) is unidentifiable under the instance-dependent noise(IDN) We propose assumption on the geometry of T(x) that "the closer two instances are, the more similar their corresponding transition matrices should be" Our method is superior to state-of-the-art approaches for label-noise learning under the challenging IDN.
  arXiv  Detail & Related papers  (2022-06-06T04:12:01Z)
• Instance-dependent Label-noise Learning under a Structural Causal Model [92.76400590283448]
  Label noise will degenerate the performance of deep learning algorithms. By leveraging a structural causal model, we propose a novel generative approach for instance-dependent label-noise learning.
  arXiv  Detail & Related papers  (2021-09-07T10:42:54Z)
• Provably End-to-end Label-Noise Learning without Anchor Points [118.97592870124937]
  We propose an end-to-end framework for solving label-noise learning without anchor points. Our proposed framework can identify the transition matrix if the clean class-posterior probabilities are sufficiently scattered.
  arXiv  Detail & Related papers  (2021-02-04T03:59:37Z)
• Extended T: Learning with Mixed Closed-set and Open-set Noisy Labels [86.5943044285146]
  The label noise transition matrix $T$ reflects the probabilities that true labels flip into noisy ones. In this paper, we focus on learning under the mixed closed-set and open-set label noise. Our method can better model the mixed label noise, following its more robust performance than the prior state-of-the-art label-noise learning methods.
  arXiv  Detail & Related papers  (2020-12-02T02:42:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.