Exploiting Embodied Simulation to Detect Novel Object Classes Through Interaction

• URL: http://arxiv.org/abs/2204.08107v1
• Date: Sun, 17 Apr 2022 23:16:55 GMT
• Title: Exploiting Embodied Simulation to Detect Novel Object Classes Through Interaction
• Authors: Nikhil Krishnaswamy, Sadaf Ghaffari
• Abstract summary: We train a reinforcement learning policy on a stacking task given a known object type, and observe the results of the agent attempting to stack various other objects based on the same trained policy. We can determine the similarity of a given object to known object types, and determine if the given object is likely dissimilar enough to the known types to be considered a novel class of object. We present the results of this method on two datasets gathered using two different policies and demonstrate what information the agent needs to extract from its environment to make these novelty judgments.
• Score: 4.507860128918788
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this paper we present a novel method for a naive agent to detect novel objects it encounters in an interaction. We train a reinforcement learning policy on a stacking task given a known object type, and then observe the results of the agent attempting to stack various other objects based on the same trained policy. By extracting embedding vectors from a convolutional neural net trained over the results of the aforementioned stacking play, we can determine the similarity of a given object to known object types, and determine if the given object is likely dissimilar enough to the known types to be considered a novel class of object. We present the results of this method on two datasets gathered using two different policies and demonstrate what information the agent needs to extract from its environment to make these novelty judgments.

Related papers

• Look Around and Learn: Self-Training Object Detection by Exploration [23.620820805804616]
  An agent learns to explore the environment using a pre-trained off-the-shelf detector to locate objects and associate pseudo-labels. By assuming that pseudo-labels for the same object must be consistent across different views, we learn the exploration policy Look Around to mine hard samples. We implement a unified benchmark of the current state-of-the-art and compare our approach with pre-existing exploration policies and perception mechanisms.
  arXiv  Detail & Related papers  (2023-02-07T16:26:45Z)
• Detecting and Accommodating Novel Types and Concepts in an Embodied Simulation Environment [4.507860128918788]
  We present methods for two types of metacognitive tasks in an AI system. We expand a neural classification model to accommodate a new category of object, and recognize when a novel object type is observed instead of misclassifying the observation as a known class. We present a suite of experiments in rapidly accommodating the introduction of new categories and concepts and in novel type detection, and an architecture to integrate the two in an interactive system.
  arXiv  Detail & Related papers  (2022-11-08T20:55:28Z)
• Spatial Reasoning for Few-Shot Object Detection [21.3564383157159]
  We propose a spatial reasoning framework that detects novel objects with only a few training examples in a context. We employ a graph convolutional network as the RoIs and their relatedness are defined as nodes and edges, respectively. We demonstrate that the proposed method significantly outperforms the state-of-the-art methods and verify its efficacy through extensive ablation studies.
  arXiv  Detail & Related papers  (2022-11-02T12:38:08Z)
• Is an Object-Centric Video Representation Beneficial for Transfer? [86.40870804449737]
  We introduce a new object-centric video recognition model on a transformer architecture. We show that the object-centric model outperforms prior video representations.
  arXiv  Detail & Related papers  (2022-07-20T17:59:44Z)
• The Familiarity Hypothesis: Explaining the Behavior of Deep Open Set Methods [86.39044549664189]
  Anomaly detection algorithms for feature-vector data identify anomalies as outliers, but outlier detection has not worked well in deep learning. This paper proposes the Familiarity Hypothesis that these methods succeed because they are detecting the absence of familiar learned features rather than the presence of novelty. The paper concludes with a discussion of whether familiarity detection is an inevitable consequence of representation learning.
  arXiv  Detail & Related papers  (2022-03-04T18:32:58Z)
• Robust Region Feature Synthesizer for Zero-Shot Object Detection [87.79902339984142]
  We build a novel zero-shot object detection framework that contains an Intra-class Semantic Diverging component and an Inter-class Structure Preserving component. It is the first study to carry out zero-shot object detection in remote sensing imagery.
  arXiv  Detail & Related papers  (2022-01-01T03:09:15Z)
• Contrastive Object Detection Using Knowledge Graph Embeddings [72.17159795485915]
  We compare the error statistics of the class embeddings learned from a one-hot approach with semantically structured embeddings from natural language processing or knowledge graphs. We propose a knowledge-embedded design for keypoint-based and transformer-based object detection architectures.
  arXiv  Detail & Related papers  (2021-12-21T17:10:21Z)
• Disentangling What and Where for 3D Object-Centric Representations Through Active Inference [4.088019409160893]
  We propose an active inference agent that can learn novel object categories over time. We show that our agent is able to learn representations for many object categories in an unsupervised way. We validate our system in an end-to-end fashion where the agent is able to search for an object at a given pose from a pixel-based rendering.
  arXiv  Detail & Related papers  (2021-08-26T12:49:07Z)
• Aligning Pretraining for Detection via Object-Level Contrastive Learning [57.845286545603415]
  Image-level contrastive representation learning has proven to be highly effective as a generic model for transfer learning. We argue that this could be sub-optimal and thus advocate a design principle which encourages alignment between the self-supervised pretext task and the downstream task. Our method, called Selective Object COntrastive learning (SoCo), achieves state-of-the-art results for transfer performance on COCO detection.
  arXiv  Detail & Related papers  (2021-06-04T17:59:52Z)
• Adaptive Object Detection with Dual Multi-Label Prediction [78.69064917947624]
  We propose a novel end-to-end unsupervised deep domain adaptation model for adaptive object detection. The model exploits multi-label prediction to reveal the object category information in each image. We introduce a prediction consistency regularization mechanism to assist object detection.
  arXiv  Detail & Related papers  (2020-03-29T04:23:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.