Parameter-Efficient Semantic Augmentation for Enhancing Open-Vocabulary Object Detection

• URL: http://arxiv.org/abs/2604.04444v1
• Date: Mon, 06 Apr 2026 05:41:33 GMT
• Title: Parameter-Efficient Semantic Augmentation for Enhancing Open-Vocabulary Object Detection
• Abstract summary: HSA-DINO is a parameter-efficient semantic augmentation framework for enhancing open-vocabulary object detection.<n>We introduce a semantic-aware router that dynamically selects the appropriate semantic augmentation strategy during inference.<n>We evaluate HSA-DINO on OV-COCO, several vertical domain datasets, and modified benchmark settings.
• Score: 33.35667602014138
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Open-vocabulary object detection (OVOD) enables models to detect any object category, including unseen ones. Benefiting from large-scale pre-training, existing OVOD methods achieve strong detection performance on general scenarios (e.g., OV-COCO) but suffer severe performance drops when transferred to downstream tasks with substantial domain shifts. This degradation stems from the scarcity and weak semantics of category labels in domain-specific task, as well as the inability of existing models to capture auxiliary semantics beyond coarse-grained category label. To address these issues, we propose HSA-DINO, a parameter-efficient semantic augmentation framework for enhancing open-vocabulary object detection. Specifically, we propose a multi-scale prompt bank that leverages image feature pyramids to capture hierarchical semantics and select domain-specific local semantic prompts, progressively enriching textual representations from coarse to fine-grained levels. Furthermore, we introduce a semantic-aware router that dynamically selects the appropriate semantic augmentation strategy during inference, thereby preventing parameter updates from degrading the generalization ability of the pre-trained OVOD model. We evaluate HSA-DINO on OV-COCO, several vertical domain datasets, and modified benchmark settings. The results show that HSA-DINO performs favorably against previous state-of-the-art methods, achieving a superior trade-off between domain adaptability and open-vocabulary generalization.

Related papers

• PAPT++: Risk-Aware Adversarial Tuning and Generation for Single Domain Generalization [99.33114297216183]
  Single domain generalization aims to learn a model from one labeled source domain that generalizes to unseen target domains.<n>We introduce PAPT++, a risk-aware adversarial generation-training framework for single domain generalization.
  arXiv  Detail & Related papers  (2026-09-04T07:54:30Z)
• ExDet: Open-Domain Open-Vocabulary Detection with Cross-modal Extrapolation and Rectification [36.08597733817782]
  open-domain open-vocabulary detection requires detectors to generalize to both novel categories and unseen domains.<n>We propose ExDet, a lightweight category-domain collaborative generalization framework for ODOVD.<n>ExDet consists of Text-Guided Extrapolation (TGE), a Detector lightweight- Rectification (DCR) module, and ExRPN.
  arXiv  Detail & Related papers  (2026-06-08T11:34:32Z)
• ModuSeg: Decoupling Object Discovery and Semantic Retrieval for Training-Free Weakly Supervised Segmentation [8.671473711194741]
  Weakly supervised semantic segmentation aims to achieve pixel-level predictions using image-level labels.<n>We present ModuSeg, a training-free weakly supervised semantic segmentation framework centered on explicitly decoupling object discovery and semantic assignment.
  arXiv  Detail & Related papers  (2026-04-08T12:38:07Z)
• GUIDED: Granular Understanding via Identification, Detection, and Discrimination for Fine-Grained Open-Vocabulary Object Detection [54.19989440021701]
  Fine-grained open-vocabulary object detection (FG-OVD) aims to detect novel object categories described by attribute-rich texts.<n>FG-OVD achieves new state-of-the-art results, demonstrating the benefits of disentangled modeling and modular optimization.
  arXiv  Detail & Related papers  (2026-03-27T22:08:11Z)
• QueryCraft: Transformer-Guided Query Initialization for Enhanced Human-Object Interaction Detection [7.030364980618468]
  We propose a novel plug-and-play HOI detection framework that incorporates semantic priors and guided feature learning.<n>Our method achieves state-of-the-art performance and strong generalizations on HICO-Det and V-COCO benchmarks.
  arXiv  Detail & Related papers  (2025-08-12T03:11:16Z)
• Style-Adaptive Detection Transformer for Single-Source Domain Generalized Object Detection [7.768332621617199]
  Single-source domain generalization aims to develop a detector using only source domain data that generalizes well to unseen target domains.<n>Existing methods are primarily CNN-based and improve robustness through data augmentation combined with feature alignment.<n>We propose Style-Adaptive DEtection TRansformer (SA-DETR), a DETR-based detector tailored for single-source domain generalization.
  arXiv  Detail & Related papers  (2025-04-29T07:38:37Z)
• Generalize or Detect? Towards Robust Semantic Segmentation Under Multiple Distribution Shifts [56.57141696245328]
  In open-world scenarios, where both novel classes and domains may exist, an ideal segmentation model should detect anomaly classes for safety. Existing methods often struggle to distinguish between domain-level and semantic-level distribution shifts.
  arXiv  Detail & Related papers  (2024-11-06T11:03:02Z)
• Source-Free Domain Adaptive Object Detection with Semantics Compensation [54.00183496587841]
  We introduce Weak-to-strong Semantics Compensation (WSCo) for strong data augmentation.<n>WSCo compensates for the class-relevant semantics that may be lost during strong augmentation on the fly.<n>WSCo can be implemented as a generic plug-in, easily integrable with any existing SFOD pipelines.
  arXiv  Detail & Related papers  (2024-10-07T23:32:06Z)
• Language-Guided Instance-Aware Domain-Adaptive Panoptic Segmentation [44.501770535446624]
  Key challenge in panoptic domain adaptation is reducing the domain gap between a labeled source and an unlabeled target domain.<n>We focus on incorporating instance-level adaptation via a novel cross-domain mixing strategy IMix.<n>We present an end-to-end model incorporating these two mechanisms called LIDAPS, achieving state-of-the-art results on all popular panoptic UDA benchmarks.
  arXiv  Detail & Related papers  (2024-04-04T20:42:49Z)
• Small Object Detection via Coarse-to-fine Proposal Generation and Imitation Learning [52.06176253457522]
  We propose a two-stage framework tailored for small object detection based on the Coarse-to-fine pipeline and Feature Imitation learning. CFINet achieves state-of-the-art performance on the large-scale small object detection benchmarks, SODA-D and SODA-A.
  arXiv  Detail & Related papers  (2023-08-18T13:13:09Z)
• CLIP the Gap: A Single Domain Generalization Approach for Object Detection [60.20931827772482]
  Single Domain Generalization tackles the problem of training a model on a single source domain so that it generalizes to any unseen target domain. We propose to leverage a pre-trained vision-language model to introduce semantic domain concepts via textual prompts. We achieve this via a semantic augmentation strategy acting on the features extracted by the detector backbone, as well as a text-based classification loss.
  arXiv  Detail & Related papers  (2023-01-13T12:01:18Z)
• Domain Adversarial Fine-Tuning as an Effective Regularizer [80.14528207465412]
  In Natural Language Processing (NLP), pretrained language models (LMs) that are transferred to downstream tasks have been recently shown to achieve state-of-the-art results. Standard fine-tuning can degrade the general-domain representations captured during pretraining. We introduce a new regularization technique, AFTER; domain Adversarial Fine-Tuning as an Effective Regularizer.
  arXiv  Detail & Related papers  (2020-09-28T14:35:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.