Augmenting Zero-Shot Detection Training with Image Labels

• URL: http://arxiv.org/abs/2306.06899v1
• Date: Mon, 12 Jun 2023 07:06:01 GMT
• Title: Augmenting Zero-Shot Detection Training with Image Labels
• Authors: Katharina Kornmeier, Ulla Scheler, Pascal Herrmann
• Abstract summary: Zero-shot detection (ZSD) is essential for real world detection use-cases, but remains a difficult task. Recent research attempts ZSD with detection models that output embeddings instead of direct class labels. We address this challenge by leveraging the CLIP embedding space in combination with image labels from ImageNet.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Zero-shot detection (ZSD), i.e., detection on classes not seen during training, is essential for real world detection use-cases, but remains a difficult task. Recent research attempts ZSD with detection models that output embeddings instead of direct class labels. To this aim, the output of the detection model must be aligned to a learned embedding space such as CLIP. However, this alignment is hindered by detection data sets which are expensive to produce compared to image classification annotations, and the resulting lack of category diversity in the training data. We address this challenge by leveraging the CLIP embedding space in combination with image labels from ImageNet. Our results show that image labels are able to better align the detector output to the embedding space and thus have a high potential for ZSD. Compared to only training on detection data, we see a significant gain by adding image label data of 3.3 mAP for the 65/15 split on COCO on the unseen classes, i.e., we more than double the gain of related work.

Related papers

• Collaborative Feature-Logits Contrastive Learning for Open-Set Semi-Supervised Object Detection [75.02249869573994]
  In open-set scenarios, the unlabeled dataset contains both in-distribution (ID) classes and out-of-distribution (OOD) classes. Applying semi-supervised detectors in such settings can lead to misclassifying OOD class as ID classes. We propose a simple yet effective method, termed Collaborative Feature-Logits Detector (CFL-Detector)
  arXiv  Detail & Related papers  (2024-11-20T02:57:35Z)
• Task Specific Pretraining with Noisy Labels for Remote Sensing Image Segmentation [18.598405597933752]
  Self-supervision provides remote sensing a tool to reduce the amount of exact, human-crafted geospatial annotations. In this work, we propose to exploit noisy semantic segmentation maps for model pretraining. The results from two datasets indicate the effectiveness of task-specific supervised pretraining with noisy labels.
  arXiv  Detail & Related papers  (2024-02-25T18:01:42Z)
• An Empirical Study of Pseudo-Labeling for Image-based 3D Object Detection [72.30883544352918]
  We investigate whether pseudo-labels can provide effective supervision for the baseline models under varying settings. We achieve 20.23 AP for moderate level on the KITTI-3D testing set without bells and whistles, improving the baseline model by 6.03 AP. We hope this work can provide insights for the image-based 3D detection community under a semi-supervised setting.
  arXiv  Detail & Related papers  (2022-08-15T12:17:46Z)
• Semantic-aware Dense Representation Learning for Remote Sensing Image Change Detection [20.761672725633936]
  Training deep learning-based change detection model heavily depends on labeled data. Recent trend is using remote sensing (RS) data to obtain in-domain representations via supervised or self-supervised learning (SSL) We propose dense semantic-aware pre-training for RS image CD via sampling multiple class-balanced points.
  arXiv  Detail & Related papers  (2022-05-27T06:08:33Z)
• Incorporating Semi-Supervised and Positive-Unlabeled Learning for Boosting Full Reference Image Quality Assessment [73.61888777504377]
  Full-reference (FR) image quality assessment (IQA) evaluates the visual quality of a distorted image by measuring its perceptual difference with pristine-quality reference. Unlabeled data can be easily collected from an image degradation or restoration process, making it encouraging to exploit unlabeled training data to boost FR-IQA performance. In this paper, we suggest to incorporate semi-supervised and positive-unlabeled (PU) learning for exploiting unlabeled data while mitigating the adverse effect of outliers.
  arXiv  Detail & Related papers  (2022-04-19T09:10:06Z)
• CaSP: Class-agnostic Semi-Supervised Pretraining for Detection and Segmentation [60.28924281991539]
  We propose a novel Class-agnostic Semi-supervised Pretraining (CaSP) framework to achieve a more favorable task-specificity balance. Using 3.6M unlabeled data, we achieve a remarkable performance gain of 4.7% over ImageNet-pretrained baseline on object detection.
  arXiv  Detail & Related papers  (2021-12-09T14:54:59Z)
• Region-level Active Learning for Cluttered Scenes [60.93811392293329]
  We introduce a new strategy that subsumes previous Image-level and Object-level approaches into a generalized, Region-level approach. We show that this approach significantly decreases labeling effort and improves rare object search on realistic data with inherent class-imbalance and cluttered scenes.
  arXiv  Detail & Related papers  (2021-08-20T14:02:38Z)
• Self-Paced Contrastive Learning for Semi-supervisedMedical Image Segmentation with Meta-labels [6.349708371894538]
  We propose to adapt contrastive learning to work with meta-label annotations. We use the meta-labels for pre-training the image encoder as well as to regularize a semi-supervised training. Results on three different medical image segmentation datasets show that our approach highly boosts the performance of a model trained on a few scans.
  arXiv  Detail & Related papers  (2021-07-29T04:30:46Z)
• Is Object Detection Necessary for Human-Object Interaction Recognition? [37.61038047282247]
  This paper revisits human-object interaction (HOI) recognition at image level without using supervisions of object location and human pose. We name it detection-free HOI recognition, in contrast to the existing detection-supervised approaches.
  arXiv  Detail & Related papers  (2021-07-27T21:15:00Z)
• G-SimCLR : Self-Supervised Contrastive Learning with Guided Projection via Pseudo Labelling [0.8164433158925593]
  In computer vision, it is evident that deep neural networks perform better in a supervised setting with a large amount of labeled data. In this work, we propose that, with the normalized temperature-scaled cross-entropy (NT-Xent) loss function, it is beneficial to not have images of the same category in the same batch. We use the latent space representation of a denoising autoencoder trained on the unlabeled dataset and cluster them with k-means to obtain pseudo labels.
  arXiv  Detail & Related papers  (2020-09-25T02:25:37Z)
• EHSOD: CAM-Guided End-to-end Hybrid-Supervised Object Detection with Cascade Refinement [53.69674636044927]
  We present EHSOD, an end-to-end hybrid-supervised object detection system. It can be trained in one shot on both fully and weakly-annotated data. It achieves comparable results on multiple object detection benchmarks with only 30% fully-annotated data.
  arXiv  Detail & Related papers  (2020-02-18T08:04:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.