Mitigating Noisy Correspondence by Geometrical Structure Consistency Learning

• URL: http://arxiv.org/abs/2405.16996v1
• Date: Mon, 27 May 2024 09:42:52 GMT
• Title: Mitigating Noisy Correspondence by Geometrical Structure Consistency Learning
• Authors: Zihua Zhao, Mengxi Chen, Tianjie Dai, Jiangchao Yao, Bo han, Ya Zhang, Yanfeng Wang,
• Abstract summary: Noisy correspondence is prevalent on human-annotated or web-crawled datasets. We introduce a Geometrical Structure Consistency (GSC) method to infer the true correspondence.
• Score: 43.75697355156703
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Noisy correspondence that refers to mismatches in cross-modal data pairs, is prevalent on human-annotated or web-crawled datasets. Prior approaches to leverage such data mainly consider the application of uni-modal noisy label learning without amending the impact on both cross-modal and intra-modal geometrical structures in multimodal learning. Actually, we find that both structures are effective to discriminate noisy correspondence through structural differences when being well-established. Inspired by this observation, we introduce a Geometrical Structure Consistency (GSC) method to infer the true correspondence. Specifically, GSC ensures the preservation of geometrical structures within and between modalities, allowing for the accurate discrimination of noisy samples based on structural differences. Utilizing these inferred true correspondence labels, GSC refines the learning of geometrical structures by filtering out the noisy samples. Experiments across four cross-modal datasets confirm that GSC effectively identifies noisy samples and significantly outperforms the current leading methods.

Related papers

• Measuring similarity between embedding spaces using induced neighborhood graphs [10.056989400384772]
  We propose a metric to evaluate the similarity between paired item representations. Our results show that accuracy in both analogy and zero-shot classification tasks correlates with the embedding similarity.
  arXiv  Detail & Related papers  (2024-11-13T15:22:33Z)
• GSSF: Generalized Structural Sparse Function for Deep Cross-modal Metric Learning [51.677086019209554]
  We propose a Generalized Structural Sparse to capture powerful relationships across modalities for pair-wise similarity learning. The distance metric delicately encapsulates two formats of diagonal and block-diagonal terms. Experiments on cross-modal and two extra uni-modal retrieval tasks have validated its superiority and flexibility.
  arXiv  Detail & Related papers  (2024-10-20T03:45:50Z)
• Disentangled Noisy Correspondence Learning [56.06801962154915]
  Cross-modal retrieval is crucial in understanding latent correspondences across modalities. DisNCL is a novel information-theoretic framework for feature Disentanglement in Noisy Correspondence Learning.
  arXiv  Detail & Related papers  (2024-08-10T09:49:55Z)
• Order-based Structure Learning with Normalizing Flows [7.972479571606131]
  Estimating causal structure of observational data is a challenging search problem that scales super-exponentially with graph size. Existing methods use continuous relaxations to make this problem computationally tractable but often restrict the data-generating process to additive noise models (ANMs) We present Order-based Structure Learning with Normalizing Flows (OSLow), a framework that relaxes these assumptions using autoregressive normalizing flows.
  arXiv  Detail & Related papers  (2023-08-14T22:17:33Z)
• Spatio-Temporal Structure Consistency for Semi-supervised Medical Image Classification [8.656046905043876]
  We propose a novel Spatio-Temporal Structure Consistent (STSC) learning framework. Specifically, a gram matrix is derived to combine the spatial structure consistency and temporal structure consistency. We show that our method outperforms state-of-the-art SSL methods.
  arXiv  Detail & Related papers  (2023-03-03T04:18:09Z)
• Differentiable and Transportable Structure Learning [73.84540901950616]
  We introduce D-Struct, which recovers transportability in the discovered structures through a novel architecture and loss function. Because D-Struct remains differentiable, our method can be easily adopted in existing differentiable architectures.
  arXiv  Detail & Related papers  (2022-06-13T17:50:53Z)
• Discriminative Supervised Subspace Learning for Cross-modal Retrieval [16.035973055257642]
  We propose a discriminative supervised subspace learning for cross-modal retrieval(DS2L) Specifically, we first construct a shared semantic graph to preserve the semantic structure within each modality. We then introduce the Hilbert-Schmidt Independence Criterion(HSIC) to preserve the consistence between feature-similarity and semantic-similarity of samples.
  arXiv  Detail & Related papers  (2022-01-26T14:27:39Z)
• Semi-supervised Domain Adaptive Structure Learning [72.01544419893628]
  Semi-supervised domain adaptation (SSDA) is a challenging problem requiring methods to overcome both 1) overfitting towards poorly annotated data and 2) distribution shift across domains. We introduce an adaptive structure learning method to regularize the cooperation of SSL and DA.
  arXiv  Detail & Related papers  (2021-12-12T06:11:16Z)
• Adaptive Hierarchical Similarity Metric Learning with Noisy Labels [138.41576366096137]
  We propose an Adaptive Hierarchical Similarity Metric Learning method. It considers two noise-insensitive information, textiti.e., class-wise divergence and sample-wise consistency. Our method achieves state-of-the-art performance compared with current deep metric learning approaches.
  arXiv  Detail & Related papers  (2021-10-29T02:12:18Z)
• CLARITY -- Comparing heterogeneous data using dissimiLARITY [0.39146761527401414]
  Many scientific questions can be phrased around whether the (dis)similarities between entities are conserved across such different data. Our method, CLARITY, quantifies consistency across datasets, identifies where inconsistencies arise, and aids in their interpretation.
  arXiv  Detail & Related papers  (2020-05-29T20:56:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.