Intra-Modal Neighbors Never Lie: Rectifying Inter-Modal Noisy Correspondence via Graph-Based Intra-Modal Reasoning

• URL: http://arxiv.org/abs/2606.04061v1
• Date: Tue, 02 Jun 2026 12:26:28 GMT
• Title: Intra-Modal Neighbors Never Lie: Rectifying Inter-Modal Noisy Correspondence via Graph-Based Intra-Modal Reasoning
• Abstract summary: Large-scale web-harvested datasets have fueled the progress of cross-modal retrieval but inevitably suffer from noisy correspondence.<n>We propose a novel framework, Intra-modal Neighbor-aware Noise Rectification (IN2R), which shifts the paradigm from searching for a substitute to synthesizing a reliable supervision target.
• Score: 35.81581893839085
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large-scale web-harvested datasets have fueled the progress of cross-modal retrieval but inevitably suffer from noisy correspondence, which severely degrades model generalization. Existing methods primarily address this by filtering out noise or seeking a substitute label, yet they predominantly remain bound by a "Discrete Selection" paradigm. We argue that relying on a single discrete proxy induces Single-Point Fragility and Discretization Error. To overcome these limitations, we propose a novel framework, Intra-modal Neighbor-aware Noise Rectification (IN2R), which shifts the paradigm from searching for a substitute to synthesizing a reliable supervision target. Leveraging the intrinsic geometric stability of intra-modal data, IN2R employs a Graph Refiner to perform relational reasoning over neighbors retrieved from a dynamic Cross-Model Memory. Instead of propagating discrete labels, our method synthesizes a continuous, soft prototype that reflects the consensus of the local semantic neighborhood, effectively rectifying inter-modal misalignment. Extensive experiments on Flickr30K, MS-COCO, and CC152K demonstrate that IN2R significantly outperforms state-of-the-art methods. Our code and pre-trained models are publicly available at https://github.com/liuyyy111/IN2R.

Related papers

• Beyond Noisy Signals: Dual-Level Denoising for Multi-modal Sequential Recommendation [6.902443075506667]
  We propose textbfDual-level textbfDenoising textbfMulti-modal textbfSequential textbfRecommendation framework.
  arXiv  Detail & Related papers  (2026-07-21T07:06:06Z)
• Provable Sparse Inversion and Token Relabel Enhanced One-shot Federated Learning with ViTs [77.42033827176806]
  One-Shot Federated Learning, where a central server learns a global model in a single communication round, has emerged as a promising paradigm.<n>We propose a novel Federated Model Inversion and Token Relabel framework, which trains the global model by fully exploiting all patches of synthetic images.
  arXiv  Detail & Related papers  (2026-05-11T15:49:08Z)
• Chain of Modality: From Static Fusion to Dynamic Orchestration in Omni-MLLMs [84.3271821505699]
  Chain of Modality (CoM) is an agentic framework that transitions multimodal fusion from passive concatenation to dynamic orchestration.<n> operating in either a training-free or a data-efficient SFT setting, CoM achieves robust and consistent generalization across diverse benchmarks.
  arXiv  Detail & Related papers  (2026-04-16T01:21:14Z)
• Neighbor-aware Instance Refining with Noisy Labels for Cross-Modal Retrieval [12.062625455647265]
  Cross-Modal Retrieval (CMR) has made significant progress in the field of multi-modal analysis.<n> CMR methods often fail to simultaneously satisfy model performance ceilings, calibration reliability, and data utilization rate.<n>We propose a novel robust cross-modal learning framework, namely Neighbor-aware Instance Refining with Noisy Labels (NIRNL)
  arXiv  Detail & Related papers  (2025-12-30T08:19:07Z)
• SoliReward: Mitigating Susceptibility to Reward Hacking and Annotation Noise in Video Generation Reward Models [53.19726629537694]
  Post-training alignment of video generation models with human preferences is a critical goal.<n>Current data collection paradigms, reliant on in-prompt pairwise annotations, suffer from labeling noise.<n>We propose SoliReward, a systematic framework for video RM training.
  arXiv  Detail & Related papers  (2025-12-17T14:28:23Z)
• Ivan-ISTD: Rethinking Cross-domain Heteroscedastic Noise Perturbations in Infrared Small Target Detection [53.689841037081834]
  Ivan-ISTD is designed to address the dual challenges of cross-domain shift and heteroscedastic noise perturbations in ISTD.<n>Ivan-ISTD demonstrates excellent robustness in cross-domain scenarios.
  arXiv  Detail & Related papers  (2025-10-14T07:48:31Z)
• NeighborRetr: Balancing Hub Centrality in Cross-Modal Retrieval [15.409022911063241]
  NeighborRetr is a novel method that balances the learning of hubs and adaptively adjusts the relations of various kinds of neighbors.<n>We show that NeighborRetr achieves state-of-the-art results on multiple cross-modal retrieval benchmarks.
  arXiv  Detail & Related papers  (2025-03-13T16:33:55Z)
• Robust Pseudo-label Learning with Neighbor Relation for Unsupervised Visible-Infrared Person Re-Identification [33.50249784731248]
  Unsupervised Visible-Infrared Person Re-identification (USVI-ReID) aims to match pedestrian images across visible and infrared modalities without any annotations. Recently, clustered pseudo-label methods have become predominant in USVI-ReID, although the inherent noise in pseudo-labels presents a significant obstacle. We design a Robust Pseudo-label Learning with Neighbor Relation (RPNR) framework to correct noisy pseudo-labels. Comprehensive experiments conducted on two widely recognized benchmarks, SYSU-MM01 and RegDB, demonstrate that RPNR outperforms the current state-of-the-art GUR with an average
  arXiv  Detail & Related papers  (2024-05-09T08:17:06Z)
• A Unified Optimal Transport Framework for Cross-Modal Retrieval with Noisy Labels [22.2715520667186]
  Cross-modal retrieval (CMR) aims to establish interaction between different modalities. This work proposes UOT-RCL, a Unified framework based on Optimal Transport (OT) for Robust Cross-modal Retrieval. Experiments on three widely-used cross-modal retrieval datasets demonstrate that our UOT-RCL surpasses the state-of-the-art approaches.
  arXiv  Detail & Related papers  (2024-03-20T10:34:40Z)
• Dynamic Weighted Combiner for Mixed-Modal Image Retrieval [8.683144453481328]
  Mixed-Modal Image Retrieval (MMIR) as a flexible search paradigm has attracted wide attention. Previous approaches always achieve limited performance, due to two critical factors. We propose a Dynamic Weighted Combiner (DWC) to tackle the above challenges.
  arXiv  Detail & Related papers  (2023-12-11T07:36:45Z)
• DeNoising-MOT: Towards Multiple Object Tracking with Severe Occlusions [52.63323657077447]
  We propose DNMOT, an end-to-end trainable DeNoising Transformer for multiple object tracking. Specifically, we augment the trajectory with noises during training and make our model learn the denoising process in an encoder-decoder architecture. We conduct extensive experiments on the MOT17, MOT20, and DanceTrack datasets, and the experimental results show that our method outperforms previous state-of-the-art methods by a clear margin.
  arXiv  Detail & Related papers  (2023-09-09T04:40:01Z)
• Attentive WaveBlock: Complementarity-enhanced Mutual Networks for Unsupervised Domain Adaptation in Person Re-identification and Beyond [97.25179345878443]
  This paper proposes a novel light-weight module, the Attentive WaveBlock (AWB) AWB can be integrated into the dual networks of mutual learning to enhance the complementarity and further depress noise in the pseudo-labels. Experiments demonstrate that the proposed method achieves state-of-the-art performance with significant improvements on multiple UDA person re-identification tasks.
  arXiv  Detail & Related papers  (2020-06-11T15:40:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.