SCENEREPLICA: Benchmarking Real-World Robot Manipulation by Creating
Replicable Scenes
- URL: http://arxiv.org/abs/2306.15620v3
- Date: Mon, 11 Mar 2024 06:20:07 GMT
- Title: SCENEREPLICA: Benchmarking Real-World Robot Manipulation by Creating
Replicable Scenes
- Authors: Ninad Khargonkar, Sai Haneesh Allu, Yangxiao Lu, Jishnu Jaykumar P,
Balakrishnan Prabhakaran, Yu Xiang
- Abstract summary: We present a new reproducible benchmark for evaluating robot manipulation in the real world, specifically focusing on pick-and-place.
Our benchmark uses the YCB objects, a commonly used dataset in the robotics community, to ensure that our results are comparable to other studies.
- Score: 5.80109297939618
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: We present a new reproducible benchmark for evaluating robot manipulation in
the real world, specifically focusing on pick-and-place. Our benchmark uses the
YCB objects, a commonly used dataset in the robotics community, to ensure that
our results are comparable to other studies. Additionally, the benchmark is
designed to be easily reproducible in the real world, making it accessible to
researchers and practitioners. We also provide our experimental results and
analyzes for model-based and model-free 6D robotic grasping on the benchmark,
where representative algorithms are evaluated for object perception, grasping
planning, and motion planning. We believe that our benchmark will be a valuable
tool for advancing the field of robot manipulation. By providing a standardized
evaluation framework, researchers can more easily compare different techniques
and algorithms, leading to faster progress in developing robot manipulation
methods.
Related papers
- IRASim: Learning Interactive Real-Robot Action Simulators [24.591694756757278]
We introduce a novel method, IRASim, to generate realistic videos of a robot arm that executes a given action trajectory.
To validate the effectiveness of our method, we create a new benchmark, IRASim Benchmark, based on three real-robot datasets.
Results show that IRASim outperforms all the baseline methods and is more preferable in human evaluations.
arXiv Detail & Related papers (2024-06-20T17:50:16Z) - Active Exploration in Bayesian Model-based Reinforcement Learning for Robot Manipulation [8.940998315746684]
We propose a model-based reinforcement learning (RL) approach for robotic arm end-tasks.
We employ Bayesian neural network models to represent, in a probabilistic way, both the belief and information encoded in the dynamic model during exploration.
Our experiments show the advantages of our Bayesian model-based RL approach, with similar quality in the results than relevant alternatives.
arXiv Detail & Related papers (2024-04-02T11:44:37Z) - Towards Inferring Users' Impressions of Robot Performance in Navigation
Scenarios [7.657890824144234]
We study the possibility of predicting people's impressions of robot behavior using non-verbal behavioral cues and machine learning techniques.
Results show that facial expressions alone provide useful information about human impressions of robot performance.
We provide guidelines for implementing these predictions models in real-world navigation scenarios.
arXiv Detail & Related papers (2023-10-17T21:12:32Z) - Self-Improving Robots: End-to-End Autonomous Visuomotor Reinforcement
Learning [54.636562516974884]
In imitation and reinforcement learning, the cost of human supervision limits the amount of data that robots can be trained on.
In this work, we propose MEDAL++, a novel design for self-improving robotic systems.
The robot autonomously practices the task by learning to both do and undo the task, simultaneously inferring the reward function from the demonstrations.
arXiv Detail & Related papers (2023-03-02T18:51:38Z) - Active Exploration for Robotic Manipulation [40.39182660794481]
This paper proposes a model-based active exploration approach that enables efficient learning in sparse-reward robotic manipulation tasks.
We evaluate our proposed algorithm in simulation and on a real robot, trained from scratch with our method.
arXiv Detail & Related papers (2022-10-23T18:07:51Z) - Human-Robot Collaboration and Machine Learning: A Systematic Review of
Recent Research [69.48907856390834]
Human-robot collaboration (HRC) is the approach that explores the interaction between a human and a robot.
This paper proposes a thorough literature review of the use of machine learning techniques in the context of HRC.
arXiv Detail & Related papers (2021-10-14T15:14:33Z) - A Framework for Efficient Robotic Manipulation [79.10407063260473]
We show that a single robotic arm can learn sparse-reward manipulation policies from pixels.
We show that, given only 10 demonstrations, a single robotic arm can learn sparse-reward manipulation policies from pixels.
arXiv Detail & Related papers (2020-12-14T22:18:39Z) - A User's Guide to Calibrating Robotics Simulators [54.85241102329546]
This paper proposes a set of benchmarks and a framework for the study of various algorithms aimed to transfer models and policies learnt in simulation to the real world.
We conduct experiments on a wide range of well known simulated environments to characterize and offer insights into the performance of different algorithms.
Our analysis can be useful for practitioners working in this area and can help make informed choices about the behavior and main properties of sim-to-real algorithms.
arXiv Detail & Related papers (2020-11-17T22:24:26Z) - Integrated Benchmarking and Design for Reproducible and Accessible
Evaluation of Robotic Agents [61.36681529571202]
We describe a new concept for reproducible robotics research that integrates development and benchmarking.
One of the central components of this setup is the Duckietown Autolab, a standardized setup that is itself relatively low-cost and reproducible.
We validate the system by analyzing the repeatability of experiments conducted using the infrastructure and show that there is low variance across different robot hardware and across different remote labs.
arXiv Detail & Related papers (2020-09-09T15:31:29Z) - Scalable Multi-Task Imitation Learning with Autonomous Improvement [159.9406205002599]
We build an imitation learning system that can continuously improve through autonomous data collection.
We leverage the robot's own trials as demonstrations for tasks other than the one that the robot actually attempted.
In contrast to prior imitation learning approaches, our method can autonomously collect data with sparse supervision for continuous improvement.
arXiv Detail & Related papers (2020-02-25T18:56:42Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.