Rearrangement: A Challenge for Embodied AI
- URL: http://arxiv.org/abs/2011.01975v1
- Date: Tue, 3 Nov 2020 19:42:32 GMT
- Title: Rearrangement: A Challenge for Embodied AI
- Authors: Dhruv Batra, Angel X. Chang, Sonia Chernova, Andrew J. Davison, Jia
Deng, Vladlen Koltun, Sergey Levine, Jitendra Malik, Igor Mordatch, Roozbeh
Mottaghi, Manolis Savva, Hao Su
- Abstract summary: We describe a framework for research and evaluation in Embodied AI.
Our proposal is based on a canonical task: Rearrangement.
We present experimental testbeds of rearrangement scenarios in four different simulation environments.
- Score: 229.8891614821016
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: We describe a framework for research and evaluation in Embodied AI. Our
proposal is based on a canonical task: Rearrangement. A standard task can focus
the development of new techniques and serve as a source of trained models that
can be transferred to other settings. In the rearrangement task, the goal is to
bring a given physical environment into a specified state. The goal state can
be specified by object poses, by images, by a description in language, or by
letting the agent experience the environment in the goal state. We characterize
rearrangement scenarios along different axes and describe metrics for
benchmarking rearrangement performance. To facilitate research and exploration,
we present experimental testbeds of rearrangement scenarios in four different
simulation environments. We anticipate that other datasets will be released and
new simulation platforms will be built to support training of rearrangement
agents and their deployment on physical systems.
Related papers
- M3Bench: Benchmarking Whole-body Motion Generation for Mobile Manipulation in 3D Scenes [66.44171200767839]
We propose M3Bench, a new benchmark of whole-body motion generation for mobile manipulation tasks.
M3Bench requires an embodied agent to understand its configuration, environmental constraints and task objectives.
M3Bench features 30k object rearrangement tasks across 119 diverse scenes, providing expert demonstrations generated by our newly developed M3BenchMaker.
arXiv Detail & Related papers (2024-10-09T08:38:21Z) - Learning Where to Look: Self-supervised Viewpoint Selection for Active Localization using Geometrical Information [68.10033984296247]
This paper explores the domain of active localization, emphasizing the importance of viewpoint selection to enhance localization accuracy.
Our contributions involve using a data-driven approach with a simple architecture designed for real-time operation, a self-supervised data training method, and the capability to consistently integrate our map into a planning framework tailored for real-world robotics applications.
arXiv Detail & Related papers (2024-07-22T12:32:09Z) - Instruct-ReID++: Towards Universal Purpose Instruction-Guided Person Re-identification [62.894790379098005]
We propose a novel instruct-ReID task that requires the model to retrieve images according to the given image or language instructions.
Instruct-ReID is the first exploration of a general ReID setting, where existing 6 ReID tasks can be viewed as special cases by assigning different instructions.
We propose a novel baseline model, IRM, with an adaptive triplet loss to handle various retrieval tasks within a unified framework.
arXiv Detail & Related papers (2024-05-28T03:35:46Z) - BEHAVIOR Vision Suite: Customizable Dataset Generation via Simulation [57.40024206484446]
We introduce the BEHAVIOR Vision Suite (BVS), a set of tools and assets to generate fully customized synthetic data for systematic evaluation of computer vision models.
BVS supports a large number of adjustable parameters at the scene level.
We showcase three example application scenarios.
arXiv Detail & Related papers (2024-05-15T17:57:56Z) - MANER: Multi-Agent Neural Rearrangement Planning of Objects in Cluttered
Environments [8.15681999722805]
This paper proposes a learning-based framework for multi-agent object rearrangement planning.
It addresses the challenges of task sequencing and path planning in complex environments.
arXiv Detail & Related papers (2023-06-10T23:53:28Z) - SE(3)-Equivariant Relational Rearrangement with Neural Descriptor Fields [39.562247503513156]
We present a method for performing tasks involving spatial relations between novel object instances in arbitrary poses.
Our framework provides a scalable way for specifying new tasks using only 5-10 demonstrations.
The method is tested on three multi-object rearrangement tasks in simulation and on a real robot.
arXiv Detail & Related papers (2022-11-17T18:55:42Z) - Spot the Difference: A Novel Task for Embodied Agents in Changing
Environments [43.52107532692226]
Embodied AI aims at creating intelligent agents that can move and operate inside an environment.
We propose Spot the Difference: a novel task for Embodied AI where the agent has access to an outdated map of the environment.
We propose an exploration policy that can take advantage of previous knowledge of the environment and identify changes in the scene faster and more effectively than existing agents.
arXiv Detail & Related papers (2022-04-18T18:30:56Z) - Predicting Stable Configurations for Semantic Placement of Novel Objects [37.18437299513799]
Our goal is to enable robots to repose previously unseen objects according to learned semantic relationships in novel environments.
We build our models and training from the ground up to be tightly integrated with our proposed planning algorithm for semantic placement of unknown objects.
Our approach enables motion planning for semantic rearrangement of unknown objects in scenes with varying geometry from only RGB-D sensing.
arXiv Detail & Related papers (2021-08-26T23:05:05Z) - FDMT: A Benchmark Dataset for Fine-grained Domain Adaptation in Machine
Translation [53.87731008029645]
We present a real-world fine-grained domain adaptation task in machine translation (FDMT)
The FDMT dataset consists of four sub-domains of information technology: autonomous vehicles, AI education, real-time networks and smart phone.
We make quantitative experiments and deep analyses in this new setting, which benchmarks the fine-grained domain adaptation task.
arXiv Detail & Related papers (2020-12-31T17:15:09Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.