OpenSlot: Mixed Open-Set Recognition with Object-Centric Learning

• URL: http://arxiv.org/abs/2407.02386v2
• Date: Sat, 04 Jan 2025 09:46:09 GMT
• Title: OpenSlot: Mixed Open-Set Recognition with Object-Centric Learning
• Authors: Xu Yin, Fei Pan, Guoyuan An, Yuchi Huo, Zixuan Xie, Sung-Eui Yoon,
• Abstract summary: Open-set recognition (OSR) studies typically assume that each image contains only one class label, with the unknown test set having a disjoint label space from the known test set. This paper introduces the mixed OSR problem, where test images contain multiple class semantics, with both known and unknown classes co-occurring in the negatives. We propose the OpenSlot framework, based on object-centric learning, which uses slot features to represent diverse class semantics and generate class predictions.
• Score: 21.933996792254998
• License:
• Abstract: Existing open-set recognition (OSR) studies typically assume that each image contains only one class label, with the unknown test set (negative) having a disjoint label space from the known test set (positive), a scenario referred to as full-label shift. This paper introduces the mixed OSR problem, where test images contain multiple class semantics, with both known and unknown classes co-occurring in the negatives, leading to a more complex super-label shift that better reflects real-world scenarios. To tackle this challenge, we propose the OpenSlot framework, based on object-centric learning, which uses slot features to represent diverse class semantics and generate class predictions. The proposed anti-noise slot (ANS) technique helps mitigate the impact of noise (invalid or background) slots during classification training, addressing the semantic misalignment between class predictions and ground truth. We evaluate OpenSlot on both mixed and conventional OSR benchmarks. Without elaborate designs, our method not only excels existing approaches in detecting super-label shifts across OSR tasks, but also achieves state-of-the-art performance on conventional benchmarks. Meanwhile, OpenSlot can localize class objects without using bounding boxes during training, demonstrating competitive performance in open-set object detection and potential for generalization.

Related papers

• Collaborative Feature-Logits Contrastive Learning for Open-Set Semi-Supervised Object Detection [75.02249869573994]
  In open-set scenarios, the unlabeled dataset contains both in-distribution (ID) classes and out-of-distribution (OOD) classes. Applying semi-supervised detectors in such settings can lead to misclassifying OOD class as ID classes. We propose a simple yet effective method, termed Collaborative Feature-Logits Detector (CFL-Detector)
  arXiv  Detail & Related papers  (2024-11-20T02:57:35Z)
• Open-Set Facial Expression Recognition [42.62439125553367]
  Facial expression recognition (FER) models are typically trained on datasets with a fixed number of seven basic classes. Recent research works point out that there are far more expressions than the basic ones. We propose the open-set FER task for the first time.
  arXiv  Detail & Related papers  (2024-01-23T05:57:50Z)
• Generalized Category Discovery with Clustering Assignment Consistency [56.92546133591019]
  Generalized category discovery (GCD) is a recently proposed open-world task. We propose a co-training-based framework that encourages clustering consistency. Our method achieves state-of-the-art performance on three generic benchmarks and three fine-grained visual recognition datasets.
  arXiv  Detail & Related papers  (2023-10-30T00:32:47Z)
• IOMatch: Simplifying Open-Set Semi-Supervised Learning with Joint Inliers and Outliers Utilization [36.102831230805755]
  In many real-world applications, unlabeled data will inevitably contain unseen-class outliers not belonging to any of the labeled classes. We introduce a novel open-set SSL framework, IOMatch, which can jointly utilize inliers and outliers, even when it is difficult to distinguish exactly between them.
  arXiv  Detail & Related papers  (2023-08-25T04:14:02Z)
• Learning Common Rationale to Improve Self-Supervised Representation for Fine-Grained Visual Recognition Problems [61.11799513362704]
  We propose learning an additional screening mechanism to identify discriminative clues commonly seen across instances and classes. We show that a common rationale detector can be learned by simply exploiting the GradCAM induced from the SSL objective.
  arXiv  Detail & Related papers  (2023-03-03T02:07:40Z)
• Open-World Object Detection via Discriminative Class Prototype Learning [4.055884768256164]
  Open-world object detection (OWOD) is a challenging problem that combines object detection with incremental learning and open-set learning. We propose a novel and efficient OWOD solution from a prototype perspective, which we call OCPL: Open-world object detection via discnative OCPL: Open-world object detection via discriminative OCPL: Open-world object detection via discriminative OCPL: Open-world object detection via discriminative OCPL: Open-world object detection via discriminative OCPL: Open-world object detection via discriminative OCPL: Open-world object detection via
  arXiv  Detail & Related papers  (2023-02-23T03:05:04Z)
• Generalized Category Discovery [148.32255950504182]
  We consider a highly general image recognition setting wherein, given a labelled and unlabelled set of images, the task is to categorize all images in the unlabelled set. Here, the unlabelled images may come from labelled classes or from novel ones. We first establish strong baselines by taking state-of-the-art algorithms from novel category discovery and adapting them for this task. We then introduce a simple yet effective semi-supervised $k$-means method to cluster the unlabelled data into seen and unseen classes.
  arXiv  Detail & Related papers  (2022-01-07T18:58:35Z)
• Conditional Variational Capsule Network for Open Set Recognition [64.18600886936557]
  In open set recognition, a classifier has to detect unknown classes that are not known at training time. Recently proposed Capsule Networks have shown to outperform alternatives in many fields, particularly in image recognition. In our proposal, during training, capsules features of the same known class are encouraged to match a pre-defined gaussian, one for each class.
  arXiv  Detail & Related papers  (2021-04-19T09:39:30Z)
• Learning Placeholders for Open-Set Recognition [38.57786747665563]
  We propose PlaceholdeRs for Open-SEt Recognition (Proser) to maintain classification performance on known classes and reject unknowns. Proser efficiently generates novel class by manifold mixup, and adaptively sets the value of reserved open-set classifier during training.
  arXiv  Detail & Related papers  (2021-03-28T09:18:15Z)
• Learning Open Set Network with Discriminative Reciprocal Points [70.28322390023546]
  Open set recognition aims to simultaneously classify samples from predefined classes and identify the rest as 'unknown' In this paper, we propose a new concept, Reciprocal Point, which is the potential representation of the extra-class space corresponding to each known category. Based on the bounded space constructed by reciprocal points, the risk of unknown is reduced through multi-category interaction.
  arXiv  Detail & Related papers  (2020-10-31T03:20:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.