WheelArm-Sim: A Manipulation and Navigation Combined Multimodal Synthetic Data Generation Simulator for Unified Control in Assistive Robotics

• URL: http://arxiv.org/abs/2601.21129v1
• Date: Thu, 29 Jan 2026 00:02:51 GMT
• Title: WheelArm-Sim: A Manipulation and Navigation Combined Multimodal Synthetic Data Generation Simulator for Unified Control in Assistive Robotics
• Abstract summary: We introduce the concept of WheelArm, an integrated cyber-physical system (CPS) that combines wheelchair and robotic arm controls.<n>Data collected from WheelArm-Sim is feasible for a data-driven machine learning model for integrated control.
• Score: 1.9655734270449854
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Wheelchairs and robotic arms enhance independent living by assisting individuals with upper-body and mobility limitations in their activities of daily living (ADLs). Although recent advancements in assistive robotics have focused on Wheelchair-Mounted Robotic Arms (WMRAs) and wheelchairs separately, integrated and unified control of the combination using machine learning models remains largely underexplored. To fill this gap, we introduce the concept of WheelArm, an integrated cyber-physical system (CPS) that combines wheelchair and robotic arm controls. Data collection is the first step toward developing WheelArm models. In this paper, we present WheelArm-Sim, a simulation framework developed in Isaac Sim for synthetic data collection. We evaluate its capability by collecting a manipulation and navigation combined multimodal dataset, comprising 13 tasks, 232 trajectories, and 67,783 samples. To demonstrate the potential of the WheelArm dataset, we implement a baseline model for action prediction in the mustard-picking task. The results illustrate that data collected from WheelArm-Sim is feasible for a data-driven machine learning model for integrated control.

Related papers

• RoboTacDex: A Dexterous Visual-Tactile-Action Dataset for Humanoid Manipulation [29.49648908136145]
  RoboTacDex is a large, multi-modal, and diverse dataset of dexterous manipulation behaviors performed with a humanoid robot.<n>Built on the publicly accessible humanoid robot Unitree G1, RoboTacDex consists of 6k trajectories covering 19 tasks, 23 skills, and interactions with 22 objects.
  arXiv  Detail & Related papers  (2026-06-30T15:39:29Z)
• Kine2Go: Kinematic dataset for the Unitree Go2 robot with diverse gaits and motions [5.199807441687141]
  We present Kine2Go, a dataset with 800 diverse gait kinematics trajectory motion data for the Unitree Go2 robot.<n>Our pipeline accepts data from various quadruped morphologies and translates them to a Go2-compatible format.
  arXiv  Detail & Related papers  (2026-06-12T13:13:53Z)
• Open-H-Embodiment: A Large-Scale Dataset for Enabling Foundation Models in Medical Robotics [131.25998619318702]
  We introduce Open-H-Embodiment, the largest open dataset of medical robotic video with synchronized kinematics to date.<n> GR00T-H is the first open foundation vision-language-action model for medical robotics.<n>We train Cosmos-H-Surgical-Simulator, the first action-conditioned world model to enable multi-embodiment surgical simulation.
  arXiv  Detail & Related papers  (2026-04-22T19:05:17Z)
• The Mini Wheelbot Dataset: High-Fidelity Data for Robot Learning [11.872449230532501]
  This paper introduces a comprehensive dynamics dataset for the Mini Wheelbot, an open-source, quasi-symmetric balancing reaction wheel unicycle.<n>The dataset provides 1 kHz synchronized data encompassing all onboard sensor readings, state estimates, ground-truth poses from a motion capture system, and third-person video logs.
  arXiv  Detail & Related papers  (2026-01-16T16:06:32Z)
• ActiveUMI: Robotic Manipulation with Active Perception from Robot-Free Human Demonstrations [32.570602111692914]
  We present ActiveUMI, a framework for a data collection system that transfers in-the-wild human demonstrations to robots capable of complex bimanual manipulation.<n>ActiveUMI couples a portable VR teleoperation kit with sensorized controllers that mirror the robot's end-effectors.<n>By recording an operator's deliberate head movements via a head-mounted display, our system learns the crucial link between visual attention and manipulation.
  arXiv  Detail & Related papers  (2025-10-02T02:44:21Z)
• HumanoidGen: Data Generation for Bimanual Dexterous Manipulation via LLM Reasoning [46.57163859424286]
  This paper presents HumanoidGen, an automated task creation and demonstration collection framework.<n>Specifically, we provide spatial annotations for both assets and dexterous hands based on the atomic operations.<n>In experiments, we create a novel benchmark with augmented scenarios to evaluate the quality of the collected data.
  arXiv  Detail & Related papers  (2025-07-01T15:04:38Z)
• RoboPearls: Editable Video Simulation for Robot Manipulation [81.18434338506621]
  RoboPearls is an editable video simulation framework for robotic manipulation.<n>Built on 3D Gaussian Splatting (3DGS), RoboPearls enables the construction of photo-realistic, view-consistent simulations.<n>We conduct extensive experiments on multiple datasets and scenes, including RLBench, COLOSSEUM, Ego4D, Open X-Embodiment, and a real-world robot.
  arXiv  Detail & Related papers  (2025-06-28T05:03:31Z)
• Data-Driven Predictive Control of Nonholonomic Robots Based on a Bilinear Koopman Realization: Data Does Not Replace Geometry [0.5277756703318045]
  This paper examines how EDMD models can be integrated into predictive controllers for nonholonomic mobile robots. Using only real-world measurement data, it is shown in both simulations and hardware experiments that the surrogate models enable high-precision predictive controllers.
  arXiv  Detail & Related papers  (2024-11-11T18:08:17Z)
• AutoRT: Embodied Foundation Models for Large Scale Orchestration of Robotic Agents [109.3804962220498]
  AutoRT is a system to scale up the deployment of operational robots in completely unseen scenarios with minimal human supervision. We demonstrate AutoRT proposing instructions to over 20 robots across multiple buildings and collecting 77k real robot episodes via both teleoperation and autonomous robot policies. We experimentally show that such "in-the-wild" data collected by AutoRT is significantly more diverse, and that AutoRT's use of LLMs allows for instruction following data collection robots that can align to human preferences.
  arXiv  Detail & Related papers  (2024-01-23T18:45:54Z)
• Robot Learning with Sensorimotor Pre-training [98.7755895548928]
  We present a self-supervised sensorimotor pre-training approach for robotics. Our model, called RPT, is a Transformer that operates on sequences of sensorimotor tokens. We find that sensorimotor pre-training consistently outperforms training from scratch, has favorable scaling properties, and enables transfer across different tasks, environments, and robots.
  arXiv  Detail & Related papers  (2023-06-16T17:58:10Z)
• V-MAO: Generative Modeling for Multi-Arm Manipulation of Articulated Objects [51.79035249464852]
  We present a framework for learning multi-arm manipulation of articulated objects. Our framework includes a variational generative model that learns contact point distribution over object rigid parts for each robot arm.
  arXiv  Detail & Related papers  (2021-11-07T02:31:09Z)
• Learning Predictive Models From Observation and Interaction [137.77887825854768]
  Learning predictive models from interaction with the world allows an agent, such as a robot, to learn about how the world works. However, learning a model that captures the dynamics of complex skills represents a major challenge. We propose a method to augment the training set with observational data of other agents, such as humans.
  arXiv  Detail & Related papers  (2019-12-30T01:10:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.