MoDeSuite: Robot Learning Task Suite for Benchmarking Mobile Manipulation with Deformable Objects

• URL: http://arxiv.org/abs/2507.21796v1
• Date: Tue, 29 Jul 2025 13:33:43 GMT
• Title: MoDeSuite: Robot Learning Task Suite for Benchmarking Mobile Manipulation with Deformable Objects
• Authors: Yuying Zhang, Kevin Sebastian Luck, Francesco Verdoja, Ville Kyrki, Joni Pajarinen,
• Abstract summary: We introduce MoDeSuite, the first task suite that addresses mobile manipulation tasks involving deformable objects.<n>Success in these tasks requires effective collaboration between the robot's base and manipulator, as well as the ability to exploit the deformability of the objects.<n>We train two state-of-the-art reinforcement learning algorithms and two imitation learning algorithms, highlighting the difficulties encountered and showing their performance in simulation.
• Score: 17.54380759165508
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Mobile manipulation is a critical capability for robots operating in diverse, real-world environments. However, manipulating deformable objects and materials remains a major challenge for existing robot learning algorithms. While various benchmarks have been proposed to evaluate manipulation strategies with rigid objects, there is still a notable lack of standardized benchmarks that address mobile manipulation tasks involving deformable objects. To address this gap, we introduce MoDeSuite, the first Mobile Manipulation Deformable Object task suite, designed specifically for robot learning. MoDeSuite consists of eight distinct mobile manipulation tasks covering both elastic objects and deformable objects, each presenting a unique challenge inspired by real-world robot applications. Success in these tasks requires effective collaboration between the robot's base and manipulator, as well as the ability to exploit the deformability of the objects. To evaluate and demonstrate the use of the proposed benchmark, we train two state-of-the-art reinforcement learning algorithms and two imitation learning algorithms, highlighting the difficulties encountered and showing their performance in simulation. Furthermore, we demonstrate the practical relevance of the suite by deploying the trained policies directly into the real world with the Spot robot, showcasing the potential for sim-to-real transfer. We expect that MoDeSuite will open a novel research domain in mobile manipulation involving deformable objects. Find more details, code, and videos at https://sites.google.com/view/modesuite/home.

Related papers

• ZeroMimic: Distilling Robotic Manipulation Skills from Web Videos [15.809468471562537]
  ZeroMimic generates image goal-conditioned skill policies for several common manipulation tasks.<n>We evaluate ZeroMimic's out-of-the-box performance in varied real-world and simulated kitchen settings.<n>To enable plug-and-play reuse of ZeroMimic policies on other task setups and robots, we release software and policy checkpoints.
  arXiv  Detail & Related papers  (2025-03-31T09:27:00Z)
• FLEX: A Framework for Learning Robot-Agnostic Force-based Skills Involving Sustained Contact Object Manipulation [9.292150395779332]
  We propose a novel framework for learning object-centric manipulation policies in force space.<n>Our method simplifies the action space, reduces unnecessary exploration, and decreases simulation overhead.<n>Our evaluations demonstrate that the method significantly outperforms baselines.
  arXiv  Detail & Related papers  (2025-03-17T17:49:47Z)
• Learning Object Properties Using Robot Proprioception via Differentiable Robot-Object Interaction [52.12746368727368]
  Differentiable simulation has become a powerful tool for system identification.<n>Our approach calibrates object properties by using information from the robot, without relying on data from the object itself.<n>We demonstrate the effectiveness of our method on a low-cost robotic platform.
  arXiv  Detail & Related papers  (2024-10-04T20:48:38Z)
• Tiny Robotics Dataset and Benchmark for Continual Object Detection [6.4036245876073234]
  We introduce a novel benchmark to evaluate the continual learning capabilities of object detection systems in tiny robotic platforms.<n>Our results highlight key challenges in developing robust and efficient continual learning strategies for object detectors in tiny robotics.
  arXiv  Detail & Related papers  (2024-09-24T16:21:27Z)
• Flow as the Cross-Domain Manipulation Interface [73.15952395641136]
  Im2Flow2Act enables robots to acquire real-world manipulation skills without the need of real-world robot training data. Im2Flow2Act comprises two components: a flow generation network and a flow-conditioned policy. We demonstrate Im2Flow2Act's capabilities in a variety of real-world tasks, including the manipulation of rigid, articulated, and deformable objects.
  arXiv  Detail & Related papers  (2024-07-21T16:15:02Z)
• Track2Act: Predicting Point Tracks from Internet Videos enables Generalizable Robot Manipulation [65.46610405509338]
  We seek to learn a generalizable goal-conditioned policy that enables zero-shot robot manipulation. Our framework,Track2Act predicts tracks of how points in an image should move in future time-steps based on a goal. We show that this approach of combining scalably learned track prediction with a residual policy enables diverse generalizable robot manipulation.
  arXiv  Detail & Related papers  (2024-05-02T17:56:55Z)
• RPMArt: Towards Robust Perception and Manipulation for Articulated Objects [56.73978941406907]
  We propose a framework towards Robust Perception and Manipulation for Articulated Objects ( RPMArt) RPMArt learns to estimate the articulation parameters and manipulate the articulation part from the noisy point cloud. We introduce an articulation-aware classification scheme to enhance its ability for sim-to-real transfer.
  arXiv  Detail & Related papers  (2024-03-24T05:55:39Z)
• Harmonic Mobile Manipulation [35.82197562695662]
  HarmonicMM is an end-to-end learning method that optimize both navigation and manipulation.<n>Our contributions include a new benchmark for mobile manipulation and the successful deployment with only RGB visual observation.
  arXiv  Detail & Related papers  (2023-12-11T18:54:42Z)
• Transferring Foundation Models for Generalizable Robotic Manipulation [82.12754319808197]
  We propose a novel paradigm that effectively leverages language-reasoning segmentation mask generated by internet-scale foundation models.<n>Our approach can effectively and robustly perceive object pose and enable sample-efficient generalization learning.<n>Demos can be found in our submitted video, and more comprehensive ones can be found in link1 or link2.
  arXiv  Detail & Related papers  (2023-06-09T07:22:12Z)
• Causal Policy Gradient for Whole-Body Mobile Manipulation [39.3461626518495]
  We introduce Causal MoMa, a new reinforcement learning framework to train policies for typical MoMa tasks. We evaluate the performance of Causal MoMa on three types of simulated robots across different MoMa tasks.
  arXiv  Detail & Related papers  (2023-05-04T23:23:47Z)
• A Differentiable Recipe for Learning Visual Non-Prehensile Planar Manipulation [63.1610540170754]
  We focus on the problem of visual non-prehensile planar manipulation. We propose a novel architecture that combines video decoding neural models with priors from contact mechanics. We find that our modular and fully differentiable architecture performs better than learning-only methods on unseen objects and motions.
  arXiv  Detail & Related papers  (2021-11-09T18:39:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.