AdaCo: Overcoming Visual Foundation Model Noise in 3D Semantic Segmentation via Adaptive Label Correction

• URL: http://arxiv.org/abs/2412.18255v1
• Date: Tue, 24 Dec 2024 08:12:31 GMT
• Title: AdaCo: Overcoming Visual Foundation Model Noise in 3D Semantic Segmentation via Adaptive Label Correction
• Authors: Pufan Zou, Shijia Zhao, Weijie Huang, Qiming Xia, Chenglu Wen, Wei Li, Cheng Wang,
• Abstract summary: We propose a novel label-free learning method, Adaptive Label Correction (AdaCo), for 3D semantic segmentation. AdaCo incorporates the Cross-modal Label Generation Module (CLGM), updating and adjusting the noisy samples within this supervision iteratively during training. Our proposed AdaCo can effectively mitigate the performance limitations of label-free learning networks in 3D semantic segmentation tasks.
• Score: 14.51758173099208
• License:
• Abstract: Recently, Visual Foundation Models (VFMs) have shown a remarkable generalization performance in 3D perception tasks. However, their effectiveness in large-scale outdoor datasets remains constrained by the scarcity of accurate supervision signals, the extensive noise caused by variable outdoor conditions, and the abundance of unknown objects. In this work, we propose a novel label-free learning method, Adaptive Label Correction (AdaCo), for 3D semantic segmentation. AdaCo first introduces the Cross-modal Label Generation Module (CLGM), providing cross-modal supervision with the formidable interpretive capabilities of the VFMs. Subsequently, AdaCo incorporates the Adaptive Noise Corrector (ANC), updating and adjusting the noisy samples within this supervision iteratively during training. Moreover, we develop an Adaptive Robust Loss (ARL) function to modulate each sample's sensitivity to noisy supervision, preventing potential underfitting issues associated with robust loss. Our proposed AdaCo can effectively mitigate the performance limitations of label-free learning networks in 3D semantic segmentation tasks. Extensive experiments on two outdoor benchmark datasets highlight the superior performance of our method.

Related papers

• Scalable Benchmarking and Robust Learning for Noise-Free Ego-Motion and 3D Reconstruction from Noisy Video [30.89206445146674]
  We aim to redefine robust ego-motion estimation and photorealistic 3D reconstruction by addressing a critical limitation: reliance on noise-free data. We tackle three core challenges: scalable data generation, comprehensive robustness, and model enhancement. We create Robust-Ego3D, a benchmark rigorously designed to expose noise-induced performance degradation.
  arXiv  Detail & Related papers  (2025-01-24T08:25:48Z)
• A Lesson in Splats: Teacher-Guided Diffusion for 3D Gaussian Splats Generation with 2D Supervision [65.33043028101471]
  We introduce a diffusion model for Gaussian Splats, SplatDiffusion, to enable generation of three-dimensional structures from single images. Existing methods rely on deterministic, feed-forward predictions, which limit their ability to handle the inherent ambiguity of 3D inference from 2D data.
  arXiv  Detail & Related papers  (2024-12-01T00:29:57Z)
• Robust Tiny Object Detection in Aerial Images amidst Label Noise [50.257696872021164]
  This study addresses the issue of tiny object detection under noisy label supervision. We propose a DeNoising Tiny Object Detector (DN-TOD), which incorporates a Class-aware Label Correction scheme. Our method can be seamlessly integrated into both one-stage and two-stage object detection pipelines.
  arXiv  Detail & Related papers  (2024-01-16T02:14:33Z)
• Generalized Face Forgery Detection via Adaptive Learning for Pre-trained Vision Transformer [54.32283739486781]
  We present a textbfForgery-aware textbfAdaptive textbfVision textbfTransformer (FA-ViT) under the adaptive learning paradigm. FA-ViT achieves 93.83% and 78.32% AUC scores on Celeb-DF and DFDC datasets in the cross-dataset evaluation.
  arXiv  Detail & Related papers  (2023-09-20T06:51:11Z)
• Augment and Criticize: Exploring Informative Samples for Semi-Supervised Monocular 3D Object Detection [64.65563422852568]
  We improve the challenging monocular 3D object detection problem with a general semi-supervised framework. We introduce a novel, simple, yet effective Augment and Criticize' framework that explores abundant informative samples from unlabeled data. The two new detectors, dubbed 3DSeMo_DLE and 3DSeMo_FLEX, achieve state-of-the-art results with remarkable improvements for over 3.5% AP_3D/BEV (Easy) on KITTI.
  arXiv  Detail & Related papers  (2023-03-20T16:28:15Z)
• Learning to Adapt to Unseen Abnormal Activities under Weak Supervision [43.40900198498228]
  We present a meta-learning framework for weakly supervised anomaly detection in videos. Our framework learns to adapt to unseen types of abnormal activities effectively when only video-level annotations of binary labels are available.
  arXiv  Detail & Related papers  (2022-03-25T12:15:44Z)
• Out-of-Domain Human Mesh Reconstruction via Dynamic Bilevel Online Adaptation [87.85851771425325]
  We consider a new problem of adapting a human mesh reconstruction model to out-of-domain streaming videos. We tackle this problem through online adaptation, gradually correcting the model bias during testing. We propose the Dynamic Bilevel Online Adaptation algorithm (DynaBOA)
  arXiv  Detail & Related papers  (2021-11-07T07:23:24Z)
• Guided Point Contrastive Learning for Semi-supervised Point Cloud Semantic Segmentation [90.2445084743881]
  We present a method for semi-supervised point cloud semantic segmentation to adopt unlabeled point clouds in training to boost the model performance. Inspired by the recent contrastive loss in self-supervised tasks, we propose the guided point contrastive loss to enhance the feature representation and model generalization ability.
  arXiv  Detail & Related papers  (2021-10-15T16:38:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.