Mutually-Aware Feature Learning for Few-Shot Object Counting

• URL: http://arxiv.org/abs/2408.09734v1
• Date: Mon, 19 Aug 2024 06:46:24 GMT
• Title: Mutually-Aware Feature Learning for Few-Shot Object Counting
• Authors: Yerim Jeon, Subeen Lee, Jihwan Kim, Jae-Pil Heo,
• Abstract summary: Few-shot object counting has garnered significant attention for its practicality as it aims to count target objects in a query image based on given exemplars without the need for additional training. We propose a novel framework, Mutually-Aware FEAture learning(MAFEA), which encodes query and exemplar features mutually aware of each other from the outset. Our model reaches a new state-of-the-art performance on the two challenging benchmarks, FSCD-LVIS and FSC-147, with a remarkably reduced degree of the target confusion problem.
• Score: 20.623402944601775
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Few-shot object counting has garnered significant attention for its practicality as it aims to count target objects in a query image based on given exemplars without the need for additional training. However, there is a shortcoming in the prevailing extract-and-match approach: query and exemplar features lack interaction during feature extraction since they are extracted unaware of each other and later correlated based on similarity. This can lead to insufficient target awareness of the extracted features, resulting in target confusion in precisely identifying the actual target when multiple class objects coexist. To address this limitation, we propose a novel framework, Mutually-Aware FEAture learning(MAFEA), which encodes query and exemplar features mutually aware of each other from the outset. By encouraging interaction between query and exemplar features throughout the entire pipeline, we can obtain target-aware features that are robust to a multi-category scenario. Furthermore, we introduce a background token to effectively associate the target region of query with exemplars and decouple its background region from them. Our extensive experiments demonstrate that our model reaches a new state-of-the-art performance on the two challenging benchmarks, FSCD-LVIS and FSC-147, with a remarkably reduced degree of the target confusion problem.

Related papers

• Counterfactual Reasoning for Multi-Label Image Classification via Patching-Based Training [84.95281245784348]
  Overemphasizing co-occurrence relationships can cause the overfitting issue of the model. We provide a causal inference framework to show that the correlative features caused by the target object and its co-occurring objects can be regarded as a mediator.
  arXiv  Detail & Related papers  (2024-04-09T13:13:24Z)
• Single-Shot and Multi-Shot Feature Learning for Multi-Object Tracking [55.13878429987136]
  We propose a simple yet effective two-stage feature learning paradigm to jointly learn single-shot and multi-shot features for different targets. Our method has achieved significant improvements on MOT17 and MOT20 datasets while reaching state-of-the-art performance on DanceTrack dataset.
  arXiv  Detail & Related papers  (2023-11-17T08:17:49Z)
• Semantics Meets Temporal Correspondence: Self-supervised Object-centric Learning in Videos [63.94040814459116]
  Self-supervised methods have shown remarkable progress in learning high-level semantics and low-level temporal correspondence. We propose a novel semantic-aware masked slot attention on top of the fused semantic features and correspondence maps. We adopt semantic- and instance-level temporal consistency as self-supervision to encourage temporally coherent object-centric representations.
  arXiv  Detail & Related papers  (2023-08-19T09:12:13Z)
• Beyond Semantics: Learning a Behavior Augmented Relevance Model with Self-supervised Learning [25.356999988217325]
  Relevance modeling aims to locate desirable items for corresponding queries. auxiliary query-item interactions extracted from user historical behavior data could provide hints to reveal users' search intents further. Our model builds multi-level co-attention for distilling coarse-grained and fine-grained semantic representations from both neighbor and target views.
  arXiv  Detail & Related papers  (2023-08-10T06:52:53Z)
• IoU-Enhanced Attention for End-to-End Task Specific Object Detection [17.617133414432836]
  R-CNN achieves promising results without densely tiled anchor boxes or grid points in the image. Due to the sparse nature and the one-to-one relation between the query and its attending region, it heavily depends on the self attention. This paper proposes to use IoU between different boxes as a prior for the value routing in self attention.
  arXiv  Detail & Related papers  (2022-09-21T14:36:18Z)
• Multi-object Tracking with a Hierarchical Single-branch Network [31.680667324595557]
  We propose an online multi-object tracking framework based on a hierarchical single-branch network. Our novel iHOIM loss function unifies the objectives of the two sub-tasks and encourages better detection performance. Experimental results on MOT16 and MOT20 datasets show that we can achieve state-of-the-art tracking performance.
  arXiv  Detail & Related papers  (2021-01-06T12:14:58Z)
• Few-shot Object Detection with Self-adaptive Attention Network for Remote Sensing Images [11.938537194408669]
  We propose a few-shot object detector which is designed for detecting novel objects provided with only a few examples. In order to fit the object detection settings, our proposed few-shot detector concentrates on the relations that lie in the level of objects instead of the full image. The experiments demonstrate the effectiveness of the proposed method in few-shot scenes.
  arXiv  Detail & Related papers  (2020-09-26T13:44:58Z)
• Object-Aware Multi-Branch Relation Networks for Spatio-Temporal Video Grounding [90.12181414070496]
  We propose a novel object-aware multi-branch relation network for object-aware relation discovery. We then propose multi-branch reasoning to capture critical object relationships between the main branch and auxiliary branches.
  arXiv  Detail & Related papers  (2020-08-16T15:39:56Z)
• A Few-Shot Sequential Approach for Object Counting [63.82757025821265]
  We introduce a class attention mechanism that sequentially attends to objects in the image and extracts their relevant features. The proposed technique is trained on point-level annotations and uses a novel loss function that disentangles class-dependent and class-agnostic aspects of the model. We present our results on a variety of object-counting/detection datasets, including FSOD and MS COCO.
  arXiv  Detail & Related papers  (2020-07-03T18:23:39Z)
• One-Shot Object Detection without Fine-Tuning [62.39210447209698]
  We introduce a two-stage model consisting of a first stage Matching-FCOS network and a second stage Structure-Aware Relation Module. We also propose novel training strategies that effectively improve detection performance. Our method exceeds the state-of-the-art one-shot performance consistently on multiple datasets.
  arXiv  Detail & Related papers  (2020-05-08T01:59:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.