Multiscale Vision Transformer With Deep Clustering-Guided Refinement for Weakly Supervised Object Localization

• URL: http://arxiv.org/abs/2312.09584v1
• Date: Fri, 15 Dec 2023 07:46:44 GMT
• Title: Multiscale Vision Transformer With Deep Clustering-Guided Refinement for Weakly Supervised Object Localization
• Authors: David Kim, Sinhae Cha, Byeongkeun Kang
• Abstract summary: This work addresses the task of weakly-supervised object localization. It comprises multiple object localization transformers that extract patch embeddings across various scales. We introduce a deep clustering-guided refinement method that further enhances localization accuracy.
• Score: 4.300577895958228
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This work addresses the task of weakly-supervised object localization. The goal is to learn object localization using only image-level class labels, which are much easier to obtain compared to bounding box annotations. This task is important because it reduces the need for labor-intensive ground-truth annotations. However, methods for object localization trained using weak supervision often suffer from limited accuracy in localization. To address this challenge and enhance localization accuracy, we propose a multiscale object localization transformer (MOLT). It comprises multiple object localization transformers that extract patch embeddings across various scales. Moreover, we introduce a deep clustering-guided refinement method that further enhances localization accuracy by utilizing separately extracted image segments. These segments are obtained by clustering pixels using convolutional neural networks. Finally, we demonstrate the effectiveness of our proposed method by conducting experiments on the publicly available ILSVRC-2012 dataset.

Related papers

• Background Activation Suppression for Weakly Supervised Object Localization and Semantic Segmentation [84.62067728093358]
  Weakly supervised object localization and semantic segmentation aim to localize objects using only image-level labels. New paradigm has emerged by generating a foreground prediction map to achieve pixel-level localization. This paper presents two astonishing experimental observations on the object localization learning process.
  arXiv  Detail & Related papers  (2023-09-22T15:44:10Z)
• Semantic-Constraint Matching Transformer for Weakly Supervised Object Localization [31.039698757869974]
  Weakly supervised object localization (WSOL) strives to learn to localize objects with only image-level supervision. Previous CNN-based methods suffer from partial activation issues, concentrating on the object's discriminative part instead of the entire entity scope. We propose a novel Semantic-Constraint Matching Network (SCMN) via a transformer to converge on the divergent activation.
  arXiv  Detail & Related papers  (2023-09-04T03:20:31Z)
• Rethinking the Localization in Weakly Supervised Object Localization [51.29084037301646]
  Weakly supervised object localization (WSOL) is one of the most popular and challenging tasks in computer vision. Recent dividing WSOL into two parts (class-agnostic object localization and object classification) has become the state-of-the-art pipeline for this task. We propose to replace SCR with a binary-class detector (BCD) for localizing multiple objects, where the detector is trained by discriminating the foreground and background.
  arXiv  Detail & Related papers  (2023-08-11T14:38:51Z)
• MOST: Multiple Object localization with Self-supervised Transformers for object discovery [97.47075050779085]
  We present Multiple Object localization with Self-supervised Transformers (MOST) MOST uses features of transformers trained using self-supervised learning to localize multiple objects in real world images. We show MOST can be used for self-supervised pre-training of object detectors, and yields consistent improvements on fully, semi-supervised object detection and unsupervised region proposal generation.
  arXiv  Detail & Related papers  (2023-04-11T17:57:27Z)
• Knowledge Combination to Learn Rotated Detection Without Rotated Annotation [53.439096583978504]
  Rotated bounding boxes drastically reduce output ambiguity of elongated objects. Despite the effectiveness, rotated detectors are not widely employed. We propose a framework that allows the model to predict precise rotated boxes.
  arXiv  Detail & Related papers  (2023-04-05T03:07:36Z)
• Spatial-Aware Token for Weakly Supervised Object Localization [137.0570026552845]
  We propose a task-specific spatial-aware token to condition localization in a weakly supervised manner. Experiments show that the proposed SAT achieves state-of-the-art performance on both CUB-200 and ImageNet, with 98.45% and 73.13% GT-known Loc.
  arXiv  Detail & Related papers  (2023-03-18T15:38:17Z)
• Constrained Sampling for Class-Agnostic Weakly Supervised Object Localization [10.542859578763068]
  Self-supervised vision transformers can generate accurate localization maps of the objects in an image. We propose leveraging the multiple maps generated by the different transformer heads to acquire pseudo-labels for training a weakly-supervised object localization model.
  arXiv  Detail & Related papers  (2022-09-09T19:58:38Z)
• Discriminative Sampling of Proposals in Self-Supervised Transformers for Weakly Supervised Object Localization [10.542859578763068]
  Self-supervised vision transformers can generate accurate localization maps of the objects in an image. We propose leveraging the multiple maps generated by the different transformer heads to acquire pseudo-labels for training a weakly-supervised object localization model.
  arXiv  Detail & Related papers  (2022-09-09T18:33:23Z)
• Weakly Supervised Object Localization via Transformer with Implicit Spatial Calibration [20.322494442959762]
  Weakly Supervised Object Localization (WSOL) has attracted much attention because of its low annotation cost in real applications. We introduce a simple yet effective Spatial Module (SCM) for accurate WSOL, incorporating semantic similarities of patch tokens and their spatial relationships into a unified diffusion model. SCM is designed as an external module of Transformer, and can be removed during inference to reduce the computation cost.
  arXiv  Detail & Related papers  (2022-07-21T12:37:15Z)
• Unsupervised Metric Relocalization Using Transform Consistency Loss [66.19479868638925]
  Training networks to perform metric relocalization traditionally requires accurate image correspondences. We propose a self-supervised solution, which exploits a key insight: localizing a query image within a map should yield the same absolute pose, regardless of the reference image used for registration. We evaluate our framework on synthetic and real-world data, showing our approach outperforms other supervised methods when a limited amount of ground-truth information is available.
  arXiv  Detail & Related papers  (2020-11-01T19:24:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.