DRAPER: Towards a Robust Robot Deployment and Reliable Evaluation for Quasi-Static Pick-and-Place Cloth-Shaping Neural Controllers

• URL: http://arxiv.org/abs/2409.15159v2
• Date: Fri, 14 Mar 2025 23:15:09 GMT
• Title: DRAPER: Towards a Robust Robot Deployment and Reliable Evaluation for Quasi-Static Pick-and-Place Cloth-Shaping Neural Controllers
• Authors: Halid Abdulrahim Kadi, Jose Alex Chandy, Luis Figueredo, Kasim Terzić, Praminda Caleb-Solly,
• Abstract summary: This study demonstrates a reliable real-world comparison of different simulation-trained neural controllers on flattening and folding tasks.<n>We introduce the DRAPER framework to enable this comprehensive study, which reliably reflects the true capabilities of these neural controllers.
• Score: 2.720296126199296
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Comparing robotic cloth-manipulation systems in a real-world setup is challenging. The fidelity gap between simulation-trained cloth neural controllers and real-world operation hinders the reliable deployment of these methods in physical trials. Inconsistent experimental setups and hardware limitations among different approaches obstruct objective evaluations. This study demonstrates a reliable real-world comparison of different simulation-trained neural controllers on both flattening and folding tasks with different types of fabrics varying in material, size, and colour. We introduce the DRAPER framework to enable this comprehensive study, which reliably reflects the true capabilities of these neural controllers. It specifically addresses real-world grasping errors, such as misgrasping and multilayer grasping, through real-world adaptations of the simulation environment to provide data trajectories that closely reflect real-world grasping scenarios. It also employs a special set of vision processing techniques to close the simulation-to-reality gap in the perception. Furthermore, it achieves robust grasping by adopting a tweezer-extended gripper and a grasping procedure. We demonstrate DRAPER's generalisability across different deep-learning methods and robotic platforms, offering valuable insights to the cloth manipulation research community.

Related papers

• Unreal Robotics Lab: A High-Fidelity Robotics Simulator with Advanced Physics and Rendering [4.760567755149477]
  This paper presents a novel simulation framework that integrates the Unreal Engine's advanced rendering capabilities with MuJoCo's high-precision physics simulation. Our approach enables realistic robotic perception while maintaining accurate physical interactions. We benchmark visual navigation and SLAM methods within our framework, demonstrating its utility for testing real-world robustness in controlled yet diverse scenarios.
  arXiv  Detail & Related papers  (2025-04-19T01:54:45Z)
• Hysteresis-Aware Neural Network Modeling and Whole-Body Reinforcement Learning Control of Soft Robots [14.02771001060961]
  We present a soft robotic system designed for surgical applications. We propose a whole-body neural network model that accurately captures and predicts the soft robot's whole-body motion. The proposed method showed strong performance in phantom-based surgical experiments.
  arXiv  Detail & Related papers  (2025-04-18T09:34:56Z)
• Robotic World Model: A Neural Network Simulator for Robust Policy Optimization in Robotics [50.191655141020505]
  This work advances model-based reinforcement learning by addressing the challenges of long-horizon prediction, error accumulation, and sim-to-real transfer. By providing a scalable and robust framework, the introduced methods pave the way for adaptive and efficient robotic systems in real-world applications.
  arXiv  Detail & Related papers  (2025-01-17T10:39:09Z)
• RoboPack: Learning Tactile-Informed Dynamics Models for Dense Packing [38.97168020979433]
  We introduce an approach that combines visual and tactile sensing for robotic manipulation by learning a neural, tactile-informed dynamics model. Our proposed framework, RoboPack, employs a recurrent graph neural network to estimate object states. We demonstrate our approach on a real robot equipped with a compliant Soft-Bubble tactile sensor on non-prehensile manipulation and dense packing tasks.
  arXiv  Detail & Related papers  (2024-07-01T16:08:37Z)
• Nonprehensile Planar Manipulation through Reinforcement Learning with Multimodal Categorical Exploration [8.343657309038285]
  Reinforcement Learning is a powerful framework for developing such robot controllers. We propose a multimodal exploration approach through categorical distributions, which enables us to train planar pushing RL policies. We show that the learned policies are robust to external disturbances and observation noise, and scale to tasks with multiple pushers.
  arXiv  Detail & Related papers  (2023-08-04T16:55:00Z)
• DMFC-GraspNet: Differentiable Multi-Fingered Robotic Grasp Generation in Cluttered Scenes [22.835683657191936]
  Multi-fingered robotic grasping can potentially perform complex object manipulation. Current techniques for multi-fingered robotic grasping frequently predict only a single grasp for each inference time. This paper proposes a differentiable multi-fingered grasp generation network (DMFC-GraspNet) with three main contributions to address this challenge.
  arXiv  Detail & Related papers  (2023-08-01T11:21:07Z)
• Transferring Foundation Models for Generalizable Robotic Manipulation [82.12754319808197]
  We propose a novel paradigm that effectively leverages language-reasoning segmentation mask generated by internet-scale foundation models. Our approach can effectively and robustly perceive object pose and enable sample-efficient generalization learning. Demos can be found in our submitted video, and more comprehensive ones can be found in link1 or link2.
  arXiv  Detail & Related papers  (2023-06-09T07:22:12Z)
• Residual Physics Learning and System Identification for Sim-to-real Transfer of Policies on Buoyancy Assisted Legged Robots [14.760426243769308]
  In this work, we demonstrate robust sim-to-real transfer of control policies on the BALLU robots via system identification. Rather than relying on standard supervised learning formulations, we utilize deep reinforcement learning to train an external force policy. We analyze the improved simulation fidelity by comparing the simulation trajectories against the real-world ones.
  arXiv  Detail & Related papers  (2023-03-16T18:49:05Z)
• Reconfigurable Data Glove for Reconstructing Physical and Virtual Grasps [100.72245315180433]
  We present a reconfigurable data glove design to capture different modes of human hand-object interactions. The glove operates in three modes for various downstream tasks with distinct features. We evaluate the system's three modes by (i) recording hand gestures and associated forces, (ii) improving manipulation fluency in VR, and (iii) producing realistic simulation effects of various tool uses.
  arXiv  Detail & Related papers  (2023-01-14T05:35:50Z)
• Real-to-Sim: Predicting Residual Errors of Robotic Systems with Sparse Data using a Learning-based Unscented Kalman Filter [65.93205328894608]
  We learn the residual errors between a dynamic and/or simulator model and the real robot. We show that with the learned residual errors, we can further close the reality gap between dynamic models, simulations, and actual hardware.
  arXiv  Detail & Related papers  (2022-09-07T15:15:12Z)
• Continual Predictive Learning from Videos [100.27176974654559]
  We study a new continual learning problem in the context of video prediction. We propose the continual predictive learning (CPL) approach, which learns a mixture world model via predictive experience replay. We construct two new benchmarks based on RoboNet and KTH, in which different tasks correspond to different physical robotic environments or human actions.
  arXiv  Detail & Related papers  (2022-04-12T08:32:26Z)
• DiSECt: A Differentiable Simulator for Parameter Inference and Control in Robotic Cutting [71.50844437057555]
  We present DiSECt: the first differentiable simulator for cutting soft materials. The simulator augments the finite element method with a continuous contact model based on signed distance fields. We show that the simulator can be calibrated to match resultant forces and fields from a state-of-the-art commercial solver.
  arXiv  Detail & Related papers  (2022-03-19T07:27:19Z)
• An in-depth experimental study of sensor usage and visual reasoning of robots navigating in real environments [20.105395754497202]
  We study the performance and reasoning capacities of real physical agents, trained in simulation and deployed to two different physical environments. We show, that for the PointGoal task, an agent pre-trained on wide variety of tasks and fine-tuned on a simulated version of the target environment can reach competitive performance without modelling any sim2real transfer.
  arXiv  Detail & Related papers  (2021-11-29T16:27:29Z)
• Nonprehensile Riemannian Motion Predictive Control [57.295751294224765]
  We introduce a novel Real-to-Sim reward analysis technique to reliably imagine and predict the outcome of taking possible actions for a real robotic platform. We produce a closed-loop controller to reactively push objects in a continuous action space. We observe that RMPC is robust in cluttered as well as occluded environments and outperforms the baselines.
  arXiv  Detail & Related papers  (2021-11-15T18:50:04Z)
• Robot Learning from Randomized Simulations: A Review [59.992761565399185]
  Deep learning has caused a paradigm shift in robotics research, favoring methods that require large amounts of data. State-of-the-art approaches learn in simulation where data generation is fast as well as inexpensive. We focus on a technique named 'domain randomization' which is a method for learning from randomized simulations.
  arXiv  Detail & Related papers  (2021-11-01T13:55:41Z)
• Simulation-based Bayesian inference for multi-fingered robotic grasping [6.677646909984405]
  Multi-fingered robotic grasping is an undeniable stepping stone to universal picking and dexterous manipulation. Yet, multi-fingered grippers remain challenging to control because of their rich nonsmooth contact dynamics or because of noise. We propose a novel simulation-based approach for full Bayesian inference based on a deep neural network surrogate of the likelihood-to-evidence ratio.
  arXiv  Detail & Related papers  (2021-09-29T08:44:36Z)
• Online Body Schema Adaptation through Cost-Sensitive Active Learning [63.84207660737483]
  The work was implemented in a simulation environment, using the 7DoF arm of the iCub robot simulator. A cost-sensitive active learning approach is used to select optimal joint configurations. The results show cost-sensitive active learning has similar accuracy to the standard active learning approach, while reducing in about half the executed movement.
  arXiv  Detail & Related papers  (2021-01-26T16:01:02Z)
• Towards Closing the Sim-to-Real Gap in Collaborative Multi-Robot Deep Reinforcement Learning [0.06554326244334865]
  We analyze how multi-agent reinforcement learning can bridge the gap to reality in distributed multi-robot systems. We introduce the effect of sensing, calibration, and accuracy mismatches in distributed reinforcement learning. We discuss on how both the different types of perturbances and how the number of agents experiencing those perturbances affect the collaborative learning effort.
  arXiv  Detail & Related papers  (2020-08-18T11:57:33Z)
• Point Cloud Based Reinforcement Learning for Sim-to-Real and Partial Observability in Visual Navigation [62.22058066456076]
  Reinforcement Learning (RL) represents powerful tools to solve complex robotic tasks. RL does not work directly in the real-world, which is known as the sim-to-real transfer problem. We propose a method that learns on an observation space constructed by point clouds and environment randomization.
  arXiv  Detail & Related papers  (2020-07-27T17:46:59Z)
• From Simulation to Real World Maneuver Execution using Deep Reinforcement Learning [69.23334811890919]
  Deep Reinforcement Learning has proved to be able to solve many control tasks in different fields, but the behavior of these systems is not always as expected when deployed in real-world scenarios. This is mainly due to the lack of domain adaptation between simulated and real-world data together with the absence of distinction between train and test datasets. We present a system based on multiple environments in which agents are trained simultaneously, evaluating the behavior of the model in different scenarios.
  arXiv  Detail & Related papers  (2020-05-13T14:22:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.