Semantic Linking Maps for Active Visual Object Search

• URL: http://arxiv.org/abs/2006.10807v1
• Date: Thu, 18 Jun 2020 18:59:44 GMT
• Title: Semantic Linking Maps for Active Visual Object Search
• Authors: Zhen Zeng, Adrian R\"ofer, Odest Chadwicke Jenkins
• Abstract summary: We exploit background knowledge about common spatial relations between landmark and target objects. We propose an active visual object search strategy method through our introduction of the Semantic Linking Maps (SLiM) model. Based on SLiM, we describe a hybrid search strategy that selects the next best view pose for searching for the target object.
• Score: 14.573513188682183
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We aim for mobile robots to function in a variety of common human environments. Such robots need to be able to reason about the locations of previously unseen target objects. Landmark objects can help this reasoning by narrowing down the search space significantly. More specifically, we can exploit background knowledge about common spatial relations between landmark and target objects. For example, seeing a table and knowing that cups can often be found on tables aids the discovery of a cup. Such correlations can be expressed as distributions over possible pairing relationships of objects. In this paper, we propose an active visual object search strategy method through our introduction of the Semantic Linking Maps (SLiM) model. SLiM simultaneously maintains the belief over a target object's location as well as landmark objects' locations, while accounting for probabilistic inter-object spatial relations. Based on SLiM, we describe a hybrid search strategy that selects the next best view pose for searching for the target object based on the maintained belief. We demonstrate the efficiency of our SLiM-based search strategy through comparative experiments in simulated environments. We further demonstrate the real-world applicability of SLiM-based search in scenarios with a Fetch mobile manipulation robot.

Related papers

• EarthVQA: Towards Queryable Earth via Relational Reasoning-Based Remote Sensing Visual Question Answering [11.37120215795946]
  We develop a multi-modal multi-task VQA dataset (EarthVQA) to advance relational reasoning-based judging, counting, and comprehensive analysis. The EarthVQA dataset contains 6000 images, corresponding semantic masks, and 208,593 QA pairs with urban and rural governance requirements embedded. We propose a Semantic OBject Awareness framework (SOBA) to advance VQA in an object-centric way.
  arXiv  Detail & Related papers  (2023-12-19T15:11:32Z)
• Object Goal Navigation with Recursive Implicit Maps [92.6347010295396]
  We propose an implicit spatial map for object goal navigation. Our method significantly outperforms the state of the art on the challenging MP3D dataset. We deploy our model on a real robot and achieve encouraging object goal navigation results in real scenes.
  arXiv  Detail & Related papers  (2023-08-10T14:21:33Z)
• An Object SLAM Framework for Association, Mapping, and High-Level Tasks [12.62957558651032]
  We present a comprehensive object SLAM framework that focuses on object-based perception and object-oriented robot tasks. A range of public datasets and real-world results have been used to evaluate the proposed object SLAM framework for its efficient performance.
  arXiv  Detail & Related papers  (2023-05-12T08:10:14Z)
• Learning-based Relational Object Matching Across Views [63.63338392484501]
  We propose a learning-based approach which combines local keypoints with novel object-level features for matching object detections between RGB images. We train our object-level matching features based on appearance and inter-frame and cross-frame spatial relations between objects in an associative graph neural network.
  arXiv  Detail & Related papers  (2023-05-03T19:36:51Z)
• Fusing Local Similarities for Retrieval-based 3D Orientation Estimation of Unseen Objects [70.49392581592089]
  We tackle the task of estimating the 3D orientation of previously-unseen objects from monocular images. We follow a retrieval-based strategy and prevent the network from learning object-specific features. Our experiments on the LineMOD, LineMOD-Occluded, and T-LESS datasets show that our method yields a significantly better generalization to unseen objects than previous works.
  arXiv  Detail & Related papers  (2022-03-16T08:53:00Z)
• Towards Optimal Correlational Object Search [25.355936023640506]
  Correlational Object Search POMDP can be solved to produce search strategies that use correlational information. We conduct experiments using AI2-THOR, a realistic simulator of household environments, as well as YOLOv5, a widely-used object detector.
  arXiv  Detail & Related papers  (2021-10-19T14:03:43Z)
• SOON: Scenario Oriented Object Navigation with Graph-based Exploration [102.74649829684617]
  The ability to navigate like a human towards a language-guided target from anywhere in a 3D embodied environment is one of the 'holy grail' goals of intelligent robots. Most visual navigation benchmarks focus on navigating toward a target from a fixed starting point, guided by an elaborate set of instructions that depicts step-by-step. This approach deviates from real-world problems in which human-only describes what the object and its surrounding look like and asks the robot to start navigation from anywhere.
  arXiv  Detail & Related papers  (2021-03-31T15:01:04Z)
• Semantic and Geometric Modeling with Neural Message Passing in 3D Scene Graphs for Hierarchical Mechanical Search [48.655167907740136]
  We use a 3D scene graph representation to capture the hierarchical, semantic, and geometric aspects of this problem. We introduce Hierarchical Mechanical Search (HMS), a method that guides an agent's actions towards finding a target object specified with a natural language description. HMS is evaluated on a novel dataset of 500 3D scene graphs with dense placements of semantically related objects in storage locations.
  arXiv  Detail & Related papers  (2020-12-07T21:04:34Z)
• Occlusion-Aware Search for Object Retrieval in Clutter [4.693170687870612]
  We address the manipulation task of retrieving a target object from a cluttered shelf. When the target object is hidden, the robot must search through the clutter for retrieving it. We present a data-driven hybrid planner for generating occlusion-aware actions in closed-loop.
  arXiv  Detail & Related papers  (2020-11-06T13:15:27Z)
• Object Goal Navigation using Goal-Oriented Semantic Exploration [98.14078233526476]
  This work studies the problem of object goal navigation which involves navigating to an instance of the given object category in unseen environments. We propose a modular system called, Goal-Oriented Semantic Exploration' which builds an episodic semantic map and uses it to explore the environment efficiently.
  arXiv  Detail & Related papers  (2020-07-01T17:52:32Z)
• Learning hierarchical relationships for object-goal navigation [7.074818959144171]
  We present Memory-utilized Joint hierarchical Object Learning for Navigation in Indoor Rooms (MJOLNIR) MJOLNIR is a target-driven navigation algorithm, which considers the inherent relationship between target objects, and the more salient contextual objects occurring in its surrounding. Our model learns to converge much faster than other algorithms, without suffering from the well-known overfitting problem.
  arXiv  Detail & Related papers  (2020-03-15T04:01:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.