Localizing Objects with Self-Supervised Transformers and no Labels

• URL: http://arxiv.org/abs/2109.14279v1
• Date: Wed, 29 Sep 2021 09:01:07 GMT
• Title: Localizing Objects with Self-Supervised Transformers and no Labels
• Authors: Oriane Sim\'eoni and Gilles Puy and Huy V. Vo and Simon Roburin and Spyros Gidaris and Andrei Bursuc and Patrick P\'erez and Renaud Marlet and Jean Ponce
• Abstract summary: Localizing objects in image collections without supervision can help to avoid expensive annotation campaigns. We propose a simple approach to this problem, that leverages the activation features of a vision transformer pre-trained in a self-supervised manner. We outperform state-of-the-art object discovery methods by up to 8 CorLoc points on PASCAL VOC 2012.
• Score: 44.364726903520086
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Localizing objects in image collections without supervision can help to avoid expensive annotation campaigns. We propose a simple approach to this problem, that leverages the activation features of a vision transformer pre-trained in a self-supervised manner. Our method, LOST, does not require any external object proposal nor any exploration of the image collection; it operates on a single image. Yet, we outperform state-of-the-art object discovery methods by up to 8 CorLoc points on PASCAL VOC 2012. We also show that training a class-agnostic detector on the discovered objects boosts results by another 7 points. Moreover, we show promising results on the unsupervised object discovery task. The code to reproduce our results can be found at https://github.com/valeoai/LOST.

Related papers

• Unsupervised Object Localization in the Era of Self-Supervised ViTs: A Survey [33.692534984177364]
  Recent works show that it is possible to perform class-agnostic unsupervised object localization by exploiting self-supervised pre-trained features. We propose here a survey of unsupervised object localization methods that discover objects in images without requiring any manual annotation.
  arXiv  Detail & Related papers  (2023-10-19T16:57:49Z)
• SalienDet: A Saliency-based Feature Enhancement Algorithm for Object Detection for Autonomous Driving [160.57870373052577]
  We propose a saliency-based OD algorithm (SalienDet) to detect unknown objects. Our SalienDet utilizes a saliency-based algorithm to enhance image features for object proposal generation. We design a dataset relabeling approach to differentiate the unknown objects from all objects in training sample set to achieve Open-World Detection.
  arXiv  Detail & Related papers  (2023-05-11T16:19:44Z)
• MOST: Multiple Object localization with Self-supervised Transformers for object discovery [97.47075050779085]
  We present Multiple Object localization with Self-supervised Transformers (MOST) MOST uses features of transformers trained using self-supervised learning to localize multiple objects in real world images. We show MOST can be used for self-supervised pre-training of object detectors, and yields consistent improvements on fully, semi-supervised object detection and unsupervised region proposal generation.
  arXiv  Detail & Related papers  (2023-04-11T17:57:27Z)
• Image Segmentation-based Unsupervised Multiple Objects Discovery [1.7674345486888503]
  Unsupervised object discovery aims to localize objects in images. We propose a fully unsupervised, bottom-up approach, for multiple objects discovery. We provide state-of-the-art results for both unsupervised class-agnostic object detection and unsupervised image segmentation.
  arXiv  Detail & Related papers  (2022-12-20T09:48:24Z)
• Unsupervised Object Localization: Observing the Background to Discover Objects [4.870509580034194]
  In this work, we take a different approach and propose to look for the background instead. This way, the salient objects emerge as a by-product without any strong assumption on what an object should be. We propose FOUND, a simple model made of a single $conv1times1$ with coarse background masks extracted from self-supervised patch-based representations.
  arXiv  Detail & Related papers  (2022-12-15T13:43:11Z)
• MOVE: Unsupervised Movable Object Segmentation and Detection [32.73565093619594]
  MOVE is a method to segment objects without any form of supervision. It exploits the fact that foreground objects can be shifted locally relative to their initial position. It gives an average CorLoc improvement of 7.2% over the SotA.
  arXiv  Detail & Related papers  (2022-10-14T16:05:46Z)
• Object Discovery via Contrastive Learning for Weakly Supervised Object Detection [12.822548027334328]
  Weakly Supervised Object Detection is a task that detects objects in an image using a model trained only on image-level annotations. We propose a novel multiple instance labeling method called object discovery. We achieve new state-of-the-art results on MS-COCO 2014 and 2017 as well as PASCAL VOC 2012, and competitive results on PASCAL VOC 2007.
  arXiv  Detail & Related papers  (2022-08-16T07:36:12Z)
• Context-Aware Transfer Attacks for Object Detection [51.65308857232767]
  We present a new approach to generate context-aware attacks for object detectors. We show that by using co-occurrence of objects and their relative locations and sizes as context information, we can successfully generate targeted mis-categorization attacks.
  arXiv  Detail & Related papers  (2021-12-06T18:26:39Z)
• Self-supervised object detection from audio-visual correspondence [101.46794879729453]
  We tackle the problem of learning object detectors without supervision. We do not assume image-level class labels, instead we extract a supervisory signal from audio-visual data. We show that our method can learn to detect generic objects that go beyond instruments, such as airplanes and cats.
  arXiv  Detail & Related papers  (2021-04-13T17:59:03Z)
• A Simple and Effective Use of Object-Centric Images for Long-Tailed Object Detection [56.82077636126353]
  We take advantage of object-centric images to improve object detection in scene-centric images. We present a simple yet surprisingly effective framework to do so. Our approach can improve the object detection (and instance segmentation) accuracy of rare objects by 50% (and 33%) relatively.
  arXiv  Detail & Related papers  (2021-02-17T17:27:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.