Training-free Boost for Open-Vocabulary Object Detection with Confidence Aggregation

• URL: http://arxiv.org/abs/2404.08603v1
• Date: Fri, 12 Apr 2024 17:02:56 GMT
• Title: Training-free Boost for Open-Vocabulary Object Detection with Confidence Aggregation
• Authors: Yanhao Zheng, Kai Liu,
• Abstract summary: Open-vocabulary object detection (OVOD) aims at localizing and recognizing visual objects from novel classes unseen at the training time. This paper systematically investigates this problem with the commonly-adopted two-stage OVOD paradigm. To alleviate this problem, this paper introduces two advanced measures to adjust confidence scores and conserve erroneously dismissed objects.
• Score: 3.0899016152680754
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Open-vocabulary object detection (OVOD) aims at localizing and recognizing visual objects from novel classes unseen at the training time. Whereas, empirical studies reveal that advanced detectors generally assign lower scores to those novel instances, which are inadvertently suppressed during inference by commonly adopted greedy strategies like Non-Maximum Suppression (NMS), leading to sub-optimal detection performance for novel classes. This paper systematically investigates this problem with the commonly-adopted two-stage OVOD paradigm. Specifically, in the region-proposal stage, proposals that contain novel instances showcase lower objectness scores, since they are treated as background proposals during the training phase. Meanwhile, in the object-classification stage, novel objects share lower region-text similarities (i.e., classification scores) due to the biased visual-language alignment by seen training samples. To alleviate this problem, this paper introduces two advanced measures to adjust confidence scores and conserve erroneously dismissed objects: (1) a class-agnostic localization quality estimate via overlap degree of region/object proposals, and (2) a text-guided visual similarity estimate with proxy prototypes for novel classes. Integrated with adjusting techniques specifically designed for the region-proposal and object-classification stages, this paper derives the aggregated confidence estimate for the open-vocabulary object detection paradigm (AggDet). Our AggDet is a generic and training-free post-processing scheme, which consistently bolsters open-vocabulary detectors across model scales and architecture designs. For instance, AggDet receives 3.3% and 1.5% gains on OV-COCO and OV-LVIS benchmarks respectively, without any training cost.

Related papers

• ACTRESS: Active Retraining for Semi-supervised Visual Grounding [52.08834188447851]
  A previous study, RefTeacher, makes the first attempt to tackle this task by adopting the teacher-student framework to provide pseudo confidence supervision and attention-based supervision. This approach is incompatible with current state-of-the-art visual grounding models, which follow the Transformer-based pipeline. Our paper proposes the ACTive REtraining approach for Semi-Supervised Visual Grounding, abbreviated as ACTRESS.
  arXiv  Detail & Related papers  (2024-07-03T16:33:31Z)
• Improving Weakly-Supervised Object Localization Using Adversarial Erasing and Pseudo Label [7.400926717561454]
  This paper investigates a framework for weakly-supervised object localization. It aims to train a neural network capable of predicting both the object class and its location using only images and their image-level class labels.
  arXiv  Detail & Related papers  (2024-04-15T06:02:09Z)
• Open-Vocabulary Object Detection with Meta Prompt Representation and Instance Contrastive Optimization [63.66349334291372]
  We propose a framework with Meta prompt and Instance Contrastive learning (MIC) schemes. Firstly, we simulate a novel-class-emerging scenario to help the prompt that learns class and background prompts generalize to novel classes. Secondly, we design an instance-level contrastive strategy to promote intra-class compactness and inter-class separation, which benefits generalization of the detector to novel class objects.
  arXiv  Detail & Related papers  (2024-03-14T14:25:10Z)
• Multi-modal Prompting for Low-Shot Temporal Action Localization [95.19505874963751]
  We consider the problem of temporal action localization under low-shot (zero-shot & few-shot) scenario. We adopt a Transformer-based two-stage action localization architecture with class-agnostic action proposal, followed by open-vocabulary classification.
  arXiv  Detail & Related papers  (2023-03-21T10:40:13Z)
• Identification of Novel Classes for Improving Few-Shot Object Detection [12.013345715187285]
  Few-shot object detection (FSOD) methods offer a remedy by realizing robust object detection using only a few training samples per class. We develop a semi-supervised algorithm to detect and then utilize unlabeled novel objects as positive samples during training to improve FSOD performance. Our experimental results indicate that our method is effective and outperforms the existing state-of-the-art (SOTA) FSOD methods.
  arXiv  Detail & Related papers  (2023-03-18T14:12:52Z)
• Learning Classifiers of Prototypes and Reciprocal Points for Universal Domain Adaptation [79.62038105814658]
  Universal Domain aims to transfer the knowledge between datasets by handling two shifts: domain-shift and categoryshift. Main challenge is correctly distinguishing the unknown target samples while adapting the distribution of known class knowledge from source to target. Most existing methods approach this problem by first training the target adapted known and then relying on the single threshold to distinguish unknown target samples.
  arXiv  Detail & Related papers  (2022-12-16T09:01:57Z)
• Open Vocabulary Object Detection with Proposal Mining and Prediction Equalization [73.14053674836838]
  Open-vocabulary object detection (OVD) aims to scale up vocabulary size to detect objects of novel categories beyond the training vocabulary. Recent work resorts to the rich knowledge in pre-trained vision-language models. We present MEDet, a novel OVD framework with proposal mining and prediction equalization.
  arXiv  Detail & Related papers  (2022-06-22T14:30:41Z)
• Object Class Aware Video Anomaly Detection through Image Translation [1.2944868613449219]
  This paper proposes a novel two-stream object-aware VAD method that learns the normal appearance and motion patterns through image translation tasks. The results show that, as significant improvements to previous methods, detections by our method are completely explainable and anomalies are localized accurately in the frames.
  arXiv  Detail & Related papers  (2022-05-03T18:04:27Z)
• Few-Shot Object Detection with Proposal Balance Refinement [21.89786914625517]
  Few-shot object detection has gained significant attention in recent years. In this paper, we analyze the lack of intersection-over-union variations induced by a biased distribution of novel samples. We present a few-shot object detection model with proposal balance refinement, a simple yet effective approach in learning object proposals.
  arXiv  Detail & Related papers  (2022-04-22T06:44:15Z)
• Plug-and-Play Few-shot Object Detection with Meta Strategy and Explicit Localization Inference [78.41932738265345]
  This paper proposes a plug detector that can accurately detect the objects of novel categories without fine-tuning process. We introduce two explicit inferences into the localization process to reduce its dependence on annotated data. It shows a significant lead in both efficiency, precision, and recall under varied evaluation protocols.
  arXiv  Detail & Related papers  (2021-10-26T03:09:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.