Multimodal Semantic-Probabilistic Objectness for Open World Object Detection

• URL: http://arxiv.org/abs/2607.23981v1
• Date: Mon, 27 Jul 2026 04:12:04 GMT
• Title: Multimodal Semantic-Probabilistic Objectness for Open World Object Detection
• Abstract summary: MSPO is a semantic calibration framework for open-world object detection.<n>It calibrates known and unknown predictions without turning OWOD into open-vocabulary classification.<n>It also improves early unknown-confusion metrics and raises PASCAL VOC final mAP by up to 2.7 points.
• Score: 2.7962583061246327
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Open-world object detection (OWOD) requires a detector to recognize known categories, discover unnamed objects from unseen categories, and incrementally learn newly annotated classes. PROB improves unknown discovery by modeling class-agnostic probabilistic objectness in the decoder-query space. However, visual objectness alone cannot determine whether an object-like query corresponds to a hard known instance, an unseen-category object, or background clutter, resulting in an ambiguous known-unknown decision boundary. We propose MSPO, a lightweight semantic calibration framework that augments PROB with task-aware known-category language priors while preserving its detector architecture and incremental learning protocol. For each currently known category, MSPO constructs an extended text description covering category attributes, visual appearance, typical scenes, and functional usage, and encodes it using a frozen CLIP text encoder. Decoder query features are projected into the same semantic space to estimate their support from the current known-category semantics. This semantic evidence is fused with PROB's visual objectness to calibrate known and unknown predictions without turning OWOD into open-vocabulary classification. Importantly, MSPO never uses future-category names, and all unseen categories remain unnamed during evaluation. Experiments on M-OWODB and S-OWODB show that MSPO improves the strong PROB baseline on the main aggregate metrics while retaining competitive unknown recall. It also improves early unknown-confusion metrics and raises PASCAL VOC final mAP by up to 2.7 points. These results demonstrate that known-category language semantics provide an effective calibration signal for probabilistic objectness under the standard OWOD setting.

Related papers

• Hyp2Former: Hierarchy-Aware Hyperbolic Embeddings for Open-Set Panoptic Segmentation [19.04466519384799]
  Open-Set Panoptic (OPS) aims to segment known thing and stuff classes while identifying valid unknown objects as separate instances.<n>Prior approaches largely treat known categories as a flat label set, ignoring the semantic hierarchy that provides valuable structural priors for distinguishing unknown objects from in-distribution classes.<n>We propose Hyp2Former, an end-to-end framework for OPS that does not require explicit modeling of unknowns during training, and instead learns hierarchical semantic similarities continuously in hyperbolic space.
  arXiv  Detail & Related papers  (2026-05-04T13:33:43Z)
• GUIDED: Granular Understanding via Identification, Detection, and Discrimination for Fine-Grained Open-Vocabulary Object Detection [54.19989440021701]
  Fine-grained open-vocabulary object detection (FG-OVD) aims to detect novel object categories described by attribute-rich texts.<n>FG-OVD achieves new state-of-the-art results, demonstrating the benefits of disentangled modeling and modular optimization.
  arXiv  Detail & Related papers  (2026-03-27T22:08:11Z)
• WeDetect: Fast Open-Vocabulary Object Detection as Retrieval [74.39703419628829]
  Open-vocabulary object detection aims to detect arbitrary classes via text prompts.<n> Methods without cross-modal fusion layers (non-fusion) offer faster inference by treating recognition as a retrieval problem.<n>We develop a model family named WeDetect to achieve state-of-the-art performance across 15 benchmarks with high inference efficiency.
  arXiv  Detail & Related papers  (2025-12-13T12:40:28Z)
• TARO: Toward Semantically Rich Open-World Object Detection [9.59690330728612]
  TARO is a novel detection framework that identifies unknown objects and classifies them into coarse parent categories within a semantic hierarchy.<n>We show TARO can categorize up to 29.9% of unknowns into meaningful coarse classes, significantly reduce confusion between unknown and known classes, and achieve competitive performance in both unknown recall and known mAP.
  arXiv  Detail & Related papers  (2025-10-10T09:15:26Z)
• A Novel Decomposed Feature-Oriented Framework for Open-Set Semantic Segmentation on LiDAR Data [6.427051055902494]
  We propose a feature-oriented framework for open-set semantic segmentation on LiDAR data.<n>We design a dual-decoder network to simultaneously perform closed-set semantic segmentation and generate distinctive features for unknown objects.<n>By integrating the results of close-set semantic segmentation and anomaly detection, we achieve effective feature-driven LiDAR open-set semantic segmentation.
  arXiv  Detail & Related papers  (2025-03-14T05:40:05Z)
• Generative Region-Language Pretraining for Open-Ended Object Detection [55.42484781608621]
  We propose a framework named GenerateU, which can detect dense objects and generate their names in a free-form way. Our framework achieves comparable results to the open-vocabulary object detection method GLIP.
  arXiv  Detail & Related papers  (2024-03-15T10:52:39Z)
• Hyp-OW: Exploiting Hierarchical Structure Learning with Hyperbolic Distance Enhances Open World Object Detection [23.005760505169803]
  Open World Object Detection is a challenging and realistic task. It involves detecting both known and unknown objects. We propose Hyp-OW, a method that learns and models hierarchical representation of known items.
  arXiv  Detail & Related papers  (2023-06-25T16:45:20Z)
• Open World DETR: Transformer based Open World Object Detection [60.64535309016623]
  We propose a two-stage training approach named Open World DETR for open world object detection based on Deformable DETR. We fine-tune the class-specific components of the model with a multi-view self-labeling strategy and a consistency constraint. Our proposed method outperforms other state-of-the-art open world object detection methods by a large margin.
  arXiv  Detail & Related papers  (2022-12-06T13:39:30Z)
• RbA: Segmenting Unknown Regions Rejected by All [1.3381749415517021]
  We show that the object queries in mask classification tend to behave like one vs all classifiers. We propose a novel outlier scoring function called RbA by defining the event of being an outlier as being rejected by all known classes. Our experiments show that mask classification improves the performance of the existing outlier detection methods, and the best results are achieved with the proposed RbA.
  arXiv  Detail & Related papers  (2022-11-25T18:50:04Z)
• OW-DETR: Open-world Detection Transformer [90.56239673123804]
  We introduce a novel end-to-end transformer-based framework, OW-DETR, for open-world object detection. OW-DETR comprises three dedicated components namely, attention-driven pseudo-labeling, novelty classification and objectness scoring. Our model outperforms the recently introduced OWOD approach, ORE, with absolute gains ranging from 1.8% to 3.3% in terms of unknown recall.
  arXiv  Detail & Related papers  (2021-12-02T18:58:30Z)
• Objects in Semantic Topology [36.297624587122506]
  A qualified open-world object detector can not only identify objects of known categories, but also discover unknown objects. We provide a unified perspective: Semantic Topology. Experiments demonstrate that semantic topology, either randomly-generated or derived from a well-trained language model, could outperform the current state-of-the-art open-world object detectors.
  arXiv  Detail & Related papers  (2021-10-06T12:15:30Z)
• Synthesizing the Unseen for Zero-shot Object Detection [72.38031440014463]
  We propose to synthesize visual features for unseen classes, so that the model learns both seen and unseen objects in the visual domain. We use a novel generative model that uses class-semantics to not only generate the features but also to discriminatively separate them.
  arXiv  Detail & Related papers  (2020-10-19T12:36:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.