Representation Handoffs for OpenArm-Based Laboratory Mobile Manipulation

• URL: http://arxiv.org/abs/2608.07154v1
• Date: Fri, 07 Aug 2026 12:19:50 GMT
• Title: Representation Handoffs for OpenArm-Based Laboratory Mobile Manipulation
• Abstract summary: This report presents an OpenArm-based mobile manipulation prototype for laboratory-style tasks.<n>It is built by integrating dual OpenArm manipulators with a mobile base, vertical slide, RGB-D sensing, lidar-based mapping, ROS2/MoveIt execution, and profile-defined skill interfaces.<n>We show how the prototype exposes missing calibration, incomplete object assets, and unfinished real-scene visual grounding as explicit deployment blockers.
• Score: 39.9928148956981
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Open-source robotics and foundation models have lowered the barrier to embodied AI, yet language-guided laboratory automation still requires reliable alignment from instructions and observations to safe actions. This field report presents an OpenArm-based mobile manipulation prototype for laboratory-style tasks, built by integrating dual OpenArm manipulators with a mobile base, vertical slide, RGB-D sensing, lidar-based mapping, ROS2/MoveIt execution, and profile-defined skill interfaces. The system is organized around representation handoffs: natural language requests are constrained into registered skill calls, sensor observations are grounded into maps and object poses, object priors provide role and skill constraints, and runtime bindings compile validated skills into executable motion goals. We use dry-run traces and startup checks to evaluate this integration path, showing how the prototype exposes missing calibration, incomplete object assets, and unfinished real-scene visual grounding as explicit deployment blockers. These intermediate representations serve as practical debugging interfaces for integrating language, perception, planning, and robot safety in embodied systems.

Related papers

• Show-Harness: Just a VLM Agent Can Play Robots [53.80887083886227]
  Embodied vision-language models (VLMs) exhibit broad intelligence about the world, yet translating this intelligence into robot control remains challenging.<n>We present Show-Harness, an Embodied Harness that enablesVLMs to "play" robots through a compact semantic interface linking intent to action.<n>We further develop GUMI, which extends the same semantic action space to GUI-based demonstration collection, allowing humans and agents to "play" robots across embodiments without specialized teleoperation hardware.
  arXiv  Detail & Related papers  (2026-09-09T17:53:38Z)
• AffordanceVLA: A Vision-Language-Action Model Empowering Action Generation through Affordance-Aware Understanding [55.535374902204296]
  AffordanceVLA is a unified framework that introduces structured affordance forecasting as a task-oriented intermediate representation.<n>We show that AffordanceVLA achieves strong performance across diverse manipulation scenarios.
  arXiv  Detail & Related papers  (2026-06-04T13:28:51Z)
• AnyUser: Translating Sketched User Intent into Domestic Robots [21.747127540075756]
  We introduce AnyUser, a unified robotic instruction system for intuitive domestic task instruction via free-form sketches on camera images.<n>AnyUser interprets multimodal inputs (sketch, vision, language) as spatial-semantic primitives to generate executable robot actions.
  arXiv  Detail & Related papers  (2026-04-06T16:16:00Z)
• Admittance-Based Motion Planning with Vision-Guided Initialization for Robotic Manipulators in Self-Driving Laboratories [0.4899818550820576]
  This paper introduces a motion-planning framework centered on admittance control to enable adaptive and compliant robotic manipulation.<n>Unlike conventional schemes, the proposed approach integrates an admittance controller directly into trajectory execution.<n>This capability enables human operators to override or redirect the robot's motion in real time.
  arXiv  Detail & Related papers  (2026-01-29T13:29:56Z)
• STORM: Slot-based Task-aware Object-centric Representation for robotic Manipulation [44.055446967139424]
  STORM is a lightweight object-centric adaptation module that augments frozen visual foundation models with semantic-aware slots for robotic manipulation.<n>Storm employs a multi-phase training strategy: object-centric slots are stabilized through visual--semantic pretraining using language embeddings, then jointly adapted with a downstream manipulation policy.<n>Experiments show that STORM improves generalization to visual distractors, and control performance compared to directly using frozen foundation model features or training object-centric representations end-to-end.
  arXiv  Detail & Related papers  (2026-01-28T08:46:04Z)
• ArtiSG: Functional 3D Scene Graph Construction via Human-demonstrated Articulated Objects Manipulation [51.54082859171464]
  ArtiSG is a framework that constructs functional 3D scene graphs by encoding human demonstrations into structured robotic memory.<n>We show that ArtiSG significantly outperforms baselines in functional element recall and articulation estimation precision.
  arXiv  Detail & Related papers  (2025-12-31T13:10:40Z)
• Executable Analytic Concepts as the Missing Link Between VLM Insight and Precise Manipulation [70.8381970762877]
  Vision-Language Models (VLMs) have demonstrated remarkable capabilities in semantic reasoning and task planning.<n>We introduce GRACE, a novel framework that grounds VLM-based reasoning through executable analytic concepts.<n>G GRACE provides a unified and interpretable interface between high-level instruction understanding and low-level robot control.
  arXiv  Detail & Related papers  (2025-10-09T09:08:33Z)
• Improving Generalization of Language-Conditioned Robot Manipulation [29.405161073483175]
  We present a framework that learns object-arrangement tasks from just a few demonstrations.<n>We validate our method on both simulation and real-world robotic environments.
  arXiv  Detail & Related papers  (2025-08-04T13:29:26Z)
• Polaris: Open-ended Interactive Robotic Manipulation via Syn2Real Visual Grounding and Large Language Models [53.22792173053473]
  We introduce an interactive robotic manipulation framework called Polaris. Polaris integrates perception and interaction by utilizing GPT-4 alongside grounded vision models. We propose a novel Synthetic-to-Real (Syn2Real) pose estimation pipeline.
  arXiv  Detail & Related papers  (2024-08-15T06:40:38Z)
• MOKA: Open-World Robotic Manipulation through Mark-Based Visual Prompting [97.52388851329667]
  We introduce Marking Open-world Keypoint Affordances (MOKA) to solve robotic manipulation tasks specified by free-form language instructions. Central to our approach is a compact point-based representation of affordance, which bridges the VLM's predictions on observed images and the robot's actions in the physical world. We evaluate and analyze MOKA's performance on various table-top manipulation tasks including tool use, deformable body manipulation, and object rearrangement.
  arXiv  Detail & Related papers  (2024-03-05T18:08:45Z)
• A System for Traded Control Teleoperation of Manipulation Tasks using Intent Prediction from Hand Gestures [20.120263332724438]
  This paper presents a teleoperation system that includes robot perception and intent prediction from hand gestures. The perception module identifies the objects present in the robot workspace and the intent prediction module which object the user likely wants to grasp.
  arXiv  Detail & Related papers  (2021-07-05T07:37:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.