TARO: Toward Semantically Rich Open-World Object Detection

• URL: http://arxiv.org/abs/2510.09173v1
• Date: Fri, 10 Oct 2025 09:15:26 GMT
• Title: TARO: Toward Semantically Rich Open-World Object Detection
• Authors: Yuchen Zhang, Yao Lu, Johannes Betz,
• Abstract summary: TARO is a novel detection framework that identifies unknown objects and classifies them into coarse parent categories within a semantic hierarchy.<n>We show TARO can categorize up to 29.9% of unknowns into meaningful coarse classes, significantly reduce confusion between unknown and known classes, and achieve competitive performance in both unknown recall and known mAP.
• Score: 9.59690330728612
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Modern object detectors are largely confined to a "closed-world" assumption, limiting them to a predefined set of classes and posing risks when encountering novel objects in real-world scenarios. While open-set detection methods aim to address this by identifying such instances as 'Unknown', this is often insufficient. Rather than treating all unknowns as a single class, assigning them more descriptive subcategories can enhance decision-making in safety-critical contexts. For example, identifying an object as an 'Unknown Animal' (requiring an urgent stop) versus 'Unknown Debris' (requiring a safe lane change) is far more useful than just 'Unknown' in autonomous driving. To bridge this gap, we introduce TARO, a novel detection framework that not only identifies unknown objects but also classifies them into coarse parent categories within a semantic hierarchy. TARO employs a unique architecture with a sparsemax-based head for modeling objectness, a hierarchy-guided relabeling component that provides auxiliary supervision, and a classification module that learns hierarchical relationships. Experiments show TARO can categorize up to 29.9% of unknowns into meaningful coarse classes, significantly reduce confusion between unknown and known classes, and achieve competitive performance in both unknown recall and known mAP. Code will be made available.

Related papers

• Knowing the Unknown: Interpretable Open-World Object Detection via Concept Decomposition Model [51.81962097623522]
  Open-world object detection (OWOD) requires incrementally detecting known categories while reliably identifying unknown objects.<n>This paper aims to make the entire OWOD framework interpretable, enabling the detector to truly "knowing the unknown"
  arXiv  Detail & Related papers  (2026-02-24T07:08:47Z)
• O1O: Grouping of Known Classes to Identify Unknown Objects as Odd-One-Out [3.637162892228131]
  Current object detection methods rely on approximate supervision with pseudo-labels corresponding to candidate locations of objects. We find that geometric cues improve unknown recall. By identifying similarities between classes within a superclass, we can identify unknown classes through an odd-one-out scoring mechanism.
  arXiv  Detail & Related papers  (2024-10-10T01:08:04Z)
• Hyp-OW: Exploiting Hierarchical Structure Learning with Hyperbolic Distance Enhances Open World Object Detection [23.005760505169803]
  Open World Object Detection is a challenging and realistic task. It involves detecting both known and unknown objects. We propose Hyp-OW, a method that learns and models hierarchical representation of known items.
  arXiv  Detail & Related papers  (2023-06-25T16:45:20Z)
• Open World DETR: Transformer based Open World Object Detection [60.64535309016623]
  We propose a two-stage training approach named Open World DETR for open world object detection based on Deformable DETR. We fine-tune the class-specific components of the model with a multi-view self-labeling strategy and a consistency constraint. Our proposed method outperforms other state-of-the-art open world object detection methods by a large margin.
  arXiv  Detail & Related papers  (2022-12-06T13:39:30Z)
• Segmenting Known Objects and Unseen Unknowns without Prior Knowledge [86.46204148650328]
  holistic segmentation aims to identify and separate objects of unseen, unknown categories into instances without any prior knowledge about them. We tackle this new problem with U3HS, which finds unknowns as highly uncertain regions and clusters their corresponding instance-aware embeddings into individual objects. Experiments on public data from MS, Cityscapes, and Lost&Found demonstrate the effectiveness of U3HS.
  arXiv  Detail & Related papers  (2022-09-12T16:59:36Z)
• UC-OWOD: Unknown-Classified Open World Object Detection [18.15975101544547]
  Open World Object Detection (OWOD) is a challenging computer vision problem. We propose a novel OWOD problem called Unknown-Classified Open World Object Detection (UC-OWOD) UC-OWOD aims to detect unknown instances and classify them into different unknown classes.
  arXiv  Detail & Related papers  (2022-07-23T08:15:30Z)
• Novel Class Discovery without Forgetting [72.52222295216062]
  We identify and formulate a new, pragmatic problem setting of NCDwF: Novel Class Discovery without Forgetting. We propose a machine learning model to incrementally discover novel categories of instances from unlabeled data. We introduce experimental protocols based on CIFAR-10, CIFAR-100 and ImageNet-1000 to measure the trade-off between knowledge retention and novel class discovery.
  arXiv  Detail & Related papers  (2022-07-21T17:54:36Z)
• More Practical Scenario of Open-set Object Detection: Open at Category Level and Closed at Super-category Level [23.98839374194848]
  Open-set object detection (OSOD) has recently attracted considerable attention. We first point out that the scenario of OSOD considered in recent studies, which considers an unlimited variety of unknown objects, has a fundamental issue. This issue leads to difficulty with the evaluation of methods' performance on unknown object detection.
  arXiv  Detail & Related papers  (2022-07-20T09:28:51Z)
• Semantic Novelty Detection via Relational Reasoning [17.660958043781154]
  We propose a novel representation learning paradigm based on relational reasoning. Our experiments show that this knowledge is directly transferable to a wide range of scenarios. It can be exploited as a plug-and-play module to convert closed-set recognition models into reliable open-set ones.
  arXiv  Detail & Related papers  (2022-07-18T15:49:27Z)
• Towards Open World Object Detection [68.79678648726416]
  ORE: Open World Object Detector is based on contrastive clustering and energy based unknown identification. We find that identifying and characterizing unknown instances helps to reduce confusion in an incremental object detection setting.
  arXiv  Detail & Related papers  (2021-03-03T18:58:18Z)
• Learning Open Set Network with Discriminative Reciprocal Points [70.28322390023546]
  Open set recognition aims to simultaneously classify samples from predefined classes and identify the rest as 'unknown' In this paper, we propose a new concept, Reciprocal Point, which is the potential representation of the extra-class space corresponding to each known category. Based on the bounded space constructed by reciprocal points, the risk of unknown is reduced through multi-category interaction.
  arXiv  Detail & Related papers  (2020-10-31T03:20:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.