Robust Object Detection with Pseudo Labels from VLMs using Per-Object Co-teaching

• URL: http://arxiv.org/abs/2511.09955v1
• Date: Fri, 14 Nov 2025 01:21:28 GMT
• Title: Robust Object Detection with Pseudo Labels from VLMs using Per-Object Co-teaching
• Abstract summary: This work introduces a novel pipeline to automatically generate pseudo-labels for training efficient, real-time object detectors.<n>Our key innovation is a per-object co-teaching-based training strategy that mitigates the inherent noise in VLM-generated labels.<n>Overall, our pipeline provides an efficient, robust, and scalable approach to train high-performance object detectors for autonomous driving.
• Score: 1.53934570513443
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Foundation models, especially vision-language models (VLMs), offer compelling zero-shot object detection for applications like autonomous driving, a domain where manual labelling is prohibitively expensive. However, their detection latency and tendency to hallucinate predictions render them unsuitable for direct deployment. This work introduces a novel pipeline that addresses this challenge by leveraging VLMs to automatically generate pseudo-labels for training efficient, real-time object detectors. Our key innovation is a per-object co-teaching-based training strategy that mitigates the inherent noise in VLM-generated labels. The proposed per-object coteaching approach filters noisy bounding boxes from training instead of filtering the entire image. Specifically, two YOLO models learn collaboratively, filtering out unreliable boxes from each mini-batch based on their peers' per-object loss values. Overall, our pipeline provides an efficient, robust, and scalable approach to train high-performance object detectors for autonomous driving, significantly reducing reliance on costly human annotation. Experimental results on the KITTI dataset demonstrate that our method outperforms a baseline YOLOv5m model, achieving a significant mAP@0.5 boost ($31.12\%$ to $46.61\%$) while maintaining real-time detection latency. Furthermore, we show that supplementing our pseudo-labelled data with a small fraction of ground truth labels ($10\%$) leads to further performance gains, reaching $57.97\%$ mAP@0.5 on the KITTI dataset. We observe similar performance improvements for the ACDC and BDD100k datasets.

Related papers

• SAM3-Assisted Training of Lightweight YOLO Models for Precision Pig Farming [0.3044748118797888]
  This work proposes a fully automated knowledge distillation pipeline to generate zero-shot pseudo-labels for training efficient YOLOv8 detectors.<n>By treating SAM 3 as an offline auto-annotator, we eliminate the manual labeling bottleneck, producing models capable of real-time inference on resource-constrained hardware.<n>Results demonstrate that a SAM 3-trained YOLOv8m achieves a mean Average Precision (mAP) of 79.4% without human intervention, while reducing inference latency by approximately 200$times$ compared to the teacher model.
  arXiv  Detail & Related papers  (2026-05-25T13:50:24Z)
• Geometry over Density: Few-Shot Cross-Domain OOD Detection [26.608445978948183]
  Out-of-distribution (OOD) detection identifies test samples that fall outside a model's training distribution.<n>We propose textbfUFCOD, a unified framework that achieves this goal through information-geometric analysis of diffusion trajectories.
  arXiv  Detail & Related papers  (2026-05-05T06:32:16Z)
• Flatter Tokens are More Valuable for Speculative Draft Model Training [8.13138934199466]
  Speculative Decoding (SD) is a key technique for accelerating Large Language Model (LLM) inference.<n>We approach this problem from a data-centric perspective, finding that not all training samples contribute equally to the SD acceptance rate.<n>We propose flatness, a new metric to quantify this property, and develop the Sample-level-flatness-based dataset Distillation (SFDD) approach.
  arXiv  Detail & Related papers  (2026-01-26T19:13:22Z)
• Streamlining the Development of Active Learning Methods in Real-World Object Detection [4.197402763771375]
  Active learning for real-world object detection faces computational and reliability challenges that limit practical deployment.<n>We introduce object-based set similarity ($mathrmOSS$), a metric that addresses these challenges.<n>This work is the first to unify AL training and evaluation strategies in object detection based on object similarity.
  arXiv  Detail & Related papers  (2025-08-27T14:10:16Z)
• Robust and Label-Efficient Deep Waste Detection [29.019461511410515]
  Effective waste sorting is critical for sustainable recycling, yet AI research in this domain continues to lag behind commercial systems.<n>In this work, we advance AI-driven waste detection by establishing strong baselines and introducing an ensemble-based semi-supervised learning framework.
  arXiv  Detail & Related papers  (2025-08-26T08:34:04Z)
• De-Simplifying Pseudo Labels to Enhancing Domain Adaptive Object Detection [33.07404672485466]
  We investigate the limitations that prevent self-labeling detectors from achieving commensurate performance with domain alignment methods.<n>We propose a novel approach called De-Simplifying Pseudo Labels (DeSimPL) to mitigate the issue.<n> Experimental results demonstrate that DeSimPL effectively reduces the proportion of simple samples during training, leading to a significant performance improvement for self-labeling detectors.
  arXiv  Detail & Related papers  (2025-07-01T09:40:27Z)
• SPARC: Score Prompting and Adaptive Fusion for Zero-Shot Multi-Label Recognition in Vision-Language Models [74.40683913645731]
  Zero-shot multi-label recognition (MLR) with Vision-Language Models (VLMs) faces significant challenges without training data, model tuning, or architectural modifications.<n>Our work proposes a novel solution treating VLMs as black boxes, leveraging scores without training data or ground truth.<n>Analysis of these prompt scores reveals VLM biases and AND''/OR' signal ambiguities, notably that maximum scores are surprisingly suboptimal compared to second-highest scores.
  arXiv  Detail & Related papers  (2025-02-24T07:15:05Z)
• Noisy Test-Time Adaptation in Vision-Language Models [73.14136220844156]
  Test-time adaptation (TTA) aims to address distribution shifts between source and target data by relying solely on target data during testing.<n>This paper introduces Zero-Shot Noisy TTA (ZS-NTTA), focusing on adapting the model to target data with noisy samples during test-time in a zero-shot manner.<n>We introduce the Adaptive Noise Detector (AdaND), which utilizes the frozen model's outputs as pseudo-labels to train a noise detector.
  arXiv  Detail & Related papers  (2025-02-20T14:37:53Z)
• DALI: Domain Adaptive LiDAR Object Detection via Distribution-level and Instance-level Pseudo Label Denoising [5.478764356647439]
  We introduce the Domain Adaptive LIdar (DALI) framework to address noise at both distribution and instance levels.<n>DALI achieves state-of-the-art results and outperforms leading approaches on most of the domain adaptation tasks.
  arXiv  Detail & Related papers  (2024-12-11T22:43:41Z)
• SIRST-5K: Exploring Massive Negatives Synthesis with Self-supervised Learning for Robust Infrared Small Target Detection [53.19618419772467]
  Single-frame infrared small target (SIRST) detection aims to recognize small targets from clutter backgrounds. With the development of Transformer, the scale of SIRST models is constantly increasing. With a rich diversity of infrared small target data, our algorithm significantly improves the model performance and convergence speed.
  arXiv  Detail & Related papers  (2024-03-08T16:14:54Z)
• Value function estimation using conditional diffusion models for control [62.27184818047923]
  We propose a simple algorithm called Diffused Value Function (DVF) It learns a joint multi-step model of the environment-robot interaction dynamics using a diffusion model. We show how DVF can be used to efficiently capture the state visitation measure for multiple controllers.
  arXiv  Detail & Related papers  (2023-06-09T18:40:55Z)
• Robust Few-shot Learning Without Using any Adversarial Samples [19.34427461937382]
  A few efforts have been made to combine the few-shot problem with the robustness objective using sophisticated Meta-Learning techniques. We propose a simple but effective alternative that does not require any adversarial samples. Inspired by the cognitive decision-making process in humans, we enforce high-level feature matching between the base class data and their corresponding low-frequency samples.
  arXiv  Detail & Related papers  (2022-11-03T05:58:26Z)
• Attentive Prototypes for Source-free Unsupervised Domain Adaptive 3D Object Detection [85.11649974840758]
  3D object detection networks tend to be biased towards the data they are trained on. We propose a single-frame approach for source-free, unsupervised domain adaptation of lidar-based 3D object detectors.
  arXiv  Detail & Related papers  (2021-11-30T18:42:42Z)
• AutoSimulate: (Quickly) Learning Synthetic Data Generation [70.82315853981838]
  We propose an efficient alternative for optimal synthetic data generation based on a novel differentiable approximation of the objective. We demonstrate that the proposed method finds the optimal data distribution faster (up to $50times$), with significantly reduced training data generation (up to $30times$) and better accuracy ($+8.7%$) on real-world test datasets than previous methods.
  arXiv  Detail & Related papers  (2020-08-16T11:36:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.