MCA: 2D-3D Retrieval with Noisy Labels via Multi-level Adaptive Correction and Alignment

• URL: http://arxiv.org/abs/2508.06104v1
• Date: Fri, 08 Aug 2025 08:06:43 GMT
• Title: MCA: 2D-3D Retrieval with Noisy Labels via Multi-level Adaptive Correction and Alignment
• Authors: Gui Zou, Chaofan Gan, Chern Hong Lim, Supavadee Aramvith, Weiyao Lin,
• Abstract summary: We propose a robust 2D-3D textbfMulti-level cross-modal adaptive textbfCorrection and textbfAlignment framework (MCA)<n>MCA achieves state-of-the-art performance on both conventional and realistic noisy 3D benchmarks.
• Score: 15.028422887133972
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: With the increasing availability of 2D and 3D data, significant advancements have been made in the field of cross-modal retrieval. Nevertheless, the existence of imperfect annotations presents considerable challenges, demanding robust solutions for 2D-3D cross-modal retrieval in the presence of noisy label conditions. Existing methods generally address the issue of noise by dividing samples independently within each modality, making them susceptible to overfitting on corrupted labels. To address these issues, we propose a robust 2D-3D \textbf{M}ulti-level cross-modal adaptive \textbf{C}orrection and \textbf{A}lignment framework (MCA). Specifically, we introduce a Multimodal Joint label Correction (MJC) mechanism that leverages multimodal historical self-predictions to jointly model the modality prediction consistency, enabling reliable label refinement. Additionally, we propose a Multi-level Adaptive Alignment (MAA) strategy to effectively enhance cross-modal feature semantics and discrimination across different levels. Extensive experiments demonstrate the superiority of our method, MCA, which achieves state-of-the-art performance on both conventional and realistic noisy 3D benchmarks, highlighting its generality and effectiveness.

Related papers

• Test-time Adaptive Hierarchical Co-enhanced Denoising Network for Reliable Multimodal Classification [55.56234913868664]
  We propose Test-time Adaptive Hierarchical Co-enhanced Denoising Network (TAHCD) for reliable learning on multimodal data.<n>The proposed method achieves superior classification performance, robustness, and generalization compared with state-of-the-art reliable multimodal learning approaches.
  arXiv  Detail & Related papers  (2026-01-12T03:14:12Z)
• Adaptive Dual Uncertainty Optimization: Boosting Monocular 3D Object Detection under Test-Time Shifts [80.32933059529135]
  Test-Time Adaptation (TTA) methods have emerged to adapt to target distributions during inference.<n>We propose Dual Uncertainty Optimization (DUO), the first TTA framework designed to jointly minimize both uncertainties for robust M3OD.<n>In parallel, we design a semantic-aware normal field constraint that preserves geometric coherence in regions with clear semantic cues.
  arXiv  Detail & Related papers  (2025-08-28T07:09:21Z)
• MoCa: Modality-aware Continual Pre-training Makes Better Bidirectional Multimodal Embeddings [75.0617088717528]
  MoCa is a framework for transforming pre-trained VLM backbones into effective bidirectional embedding models.<n>MoCa consistently improves performance across MMEB and ViDoRe-v2 benchmarks, achieving new state-of-the-art results.
  arXiv  Detail & Related papers  (2025-06-29T06:41:00Z)
• GEAL: Generalizable 3D Affordance Learning with Cross-Modal Consistency [50.11520458252128]
  Existing 3D affordance learning methods struggle with generalization and robustness due to limited annotated data.<n>We propose GEAL, a novel framework designed to enhance the generalization and robustness of 3D affordance learning by leveraging large-scale pre-trained 2D models.<n>GEAL consistently outperforms existing methods across seen and novel object categories, as well as corrupted data.
  arXiv  Detail & Related papers  (2024-12-12T17:59:03Z)
• Robust Modality-incomplete Anomaly Detection: A Modality-instructive Framework with Benchmark [69.02666229531322]
  We introduce a pioneering study that investigates Modality-Incomplete Industrial Anomaly Detection (MIIAD)<n>We find that most existing MIAD methods perform poorly on the MIIAD Bench, leading to significant performance degradation.<n>We propose a novel two-stage Robust modAlity-aware fusing and Detecting framewoRk, abbreviated as RADAR.
  arXiv  Detail & Related papers  (2024-10-02T16:47:55Z)
• DAC: 2D-3D Retrieval with Noisy Labels via Divide-and-Conquer Alignment and Correction [26.164120380820307]
  We propose a Divide-and-conquer 2D-3D cross-modal Alignment and Correction framework, which comprises Multimodal Dynamic Division (MDD) and Adaptive Alignment and Correction (AAC) In AAC, samples in distinct subsets are exploited with different alignment strategies to fully enhance the semantic compactness and meanwhile over-fitting to noisy labels. To evaluate the effectiveness in real-world scenarios, we introduce a challenging noisy benchmark, namely. N200, which comprises 200k-level samples annotated with 1156 realistic noisy labels.
  arXiv  Detail & Related papers  (2024-07-25T05:18:18Z)
• Dynamic Weighted Combiner for Mixed-Modal Image Retrieval [8.683144453481328]
  Mixed-Modal Image Retrieval (MMIR) as a flexible search paradigm has attracted wide attention. Previous approaches always achieve limited performance, due to two critical factors. We propose a Dynamic Weighted Combiner (DWC) to tackle the above challenges.
  arXiv  Detail & Related papers  (2023-12-11T07:36:45Z)
• Cross-BERT for Point Cloud Pretraining [61.762046503448936]
  We propose a new cross-modal BERT-style self-supervised learning paradigm, called Cross-BERT. To facilitate pretraining for irregular and sparse point clouds, we design two self-supervised tasks to boost cross-modal interaction. Our work highlights the effectiveness of leveraging cross-modal 2D knowledge to strengthen 3D point cloud representation and the transferable capability of BERT across modalities.
  arXiv  Detail & Related papers  (2023-12-08T08:18:12Z)
• Revisiting Domain-Adaptive 3D Object Detection by Reliable, Diverse and Class-balanced Pseudo-Labeling [38.07637524378327]
  Unsupervised domain adaptation (DA) with the aid of pseudo labeling techniques has emerged as a crucial approach for domain-adaptive 3D object detection. Existing DA methods suffer from a substantial drop in performance when applied to a multi-class training setting. We propose a novel ReDB framework tailored for learning to detect all classes at once.
  arXiv  Detail & Related papers  (2023-07-16T04:34:11Z)
• Uncertainty-Aware Adaptation for Self-Supervised 3D Human Pose Estimation [70.32536356351706]
  We introduce MRP-Net that constitutes a common deep network backbone with two output heads subscribing to two diverse configurations. We derive suitable measures to quantify prediction uncertainty at both pose and joint level. We present a comprehensive evaluation of the proposed approach and demonstrate state-of-the-art performance on benchmark datasets.
  arXiv  Detail & Related papers  (2022-03-29T07:14:58Z)
• Out-of-Domain Human Mesh Reconstruction via Dynamic Bilevel Online Adaptation [87.85851771425325]
  We consider a new problem of adapting a human mesh reconstruction model to out-of-domain streaming videos. We tackle this problem through online adaptation, gradually correcting the model bias during testing. We propose the Dynamic Bilevel Online Adaptation algorithm (DynaBOA)
  arXiv  Detail & Related papers  (2021-11-07T07:23:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.