In-the-Wild Compliant Manipulation with UMI-FT

• URL: http://arxiv.org/abs/2601.09988v1
• Date: Thu, 15 Jan 2026 02:00:03 GMT
• Title: In-the-Wild Compliant Manipulation with UMI-FT
• Abstract summary: UMI-FT is a handheld data-collection platform that mounts compact, six-axis force/torque sensors on each finger.<n>We train an adaptive compliance policy that predicts position targets, grasp force, and stiffness for execution on standard compliance controllers.
• Score: 18.632109897570835
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Many manipulation tasks require careful force modulation. With insufficient force the task may fail, while excessive force could cause damage. The high cost, bulky size and fragility of commercial force/torque (F/T) sensors have limited large-scale, force-aware policy learning. We introduce UMI-FT, a handheld data-collection platform that mounts compact, six-axis force/torque sensors on each finger, enabling finger-level wrench measurements alongside RGB, depth, and pose. Using the multimodal data collected from this device, we train an adaptive compliance policy that predicts position targets, grasp force, and stiffness for execution on standard compliance controllers. In evaluations on three contact-rich, force-sensitive tasks (whiteboard wiping, skewering zucchini, and lightbulb insertion), UMI-FT enables policies that reliably regulate external contact forces and internal grasp forces, outperforming baselines that lack compliance or force sensing. UMI-FT offers a scalable path to learning compliant manipulation from in-the-wild demonstrations. We open-source the hardware and software to facilitate broader adoption at:https://umi-ft.github.io/.

Related papers

• Universal Manipulation Exoskeleton: Learning Compliant Whole-body Policies with Real-time Torque Feedback [23.21469125720761]
  We present Universal Manipulation Exoskeleton (UME), an upper-limb exoskeleton that provides real-time haptic torque feedback.<n>We demonstrate that this combination of capabilities enables learning bimanual, whole-body, and active compliant policies that operate effectively in highly constrained spaces.
  arXiv  Detail & Related papers  (2026-06-12T07:57:13Z)
• FACTR 2: Learning External Force Sensing for Commodity Robot Arms Improves Policy Learning [54.93624123794711]
  Contact-rich manipulation requires force sensitivity, but many robot arms lack dedicated force sensors due to their high cost.<n>We present Neural External Torque Estimation (torque), a data-driven method that estimates external joint torques without needing any dedicated force sensors.<n>Next enables force-aware teleoperation on low-cost arms and improves policy learning through Force-Informed Re-Sampling Training.
  arXiv  Detail & Related papers  (2026-06-10T17:59:35Z)
• ForceFlow: Learning to Feel and Act via Contact-Driven Flow Matching [53.30290192030814]
  ForceFlow is a force-aware reactive framework built upon flow matching.<n>We show that ForceFlow achieves a 37% success rate improvement over the strong baseline ForceVLA.<n>It also exhibits accurate force signal prediction and superior performance in contact force self-regulation.
  arXiv  Detail & Related papers  (2026-05-11T13:27:00Z)
• ForceVLA2: Unleashing Hybrid Force-Position Control with Force Awareness for Contact-Rich Manipulation [75.39847750457626]
  ForceVLA2, an end-to-end vision-language-action framework, equips robots with hybrid force-position control and explicit force awareness.<n>To support learning and evaluation, we construct ForceVLA2-Dataset, containing 1,000 trajectories over 5 contact-rich tasks.<n>Experiments show that ForceVLA2 substantially improves success rates and reliability in contact-rich manipulation.
  arXiv  Detail & Related papers  (2026-03-16T12:03:58Z)
• Direction Matters: Learning Force Direction Enables Sim-to-Real Contact-Rich Manipulation [28.707088673008172]
  We propose a framework that leverages expert-designed controller logic for transfer.<n>Inspired by the success of privileged supervision in tasks, we employ a human-designed finite state machine based position/force controller in simulation.<n>Unlike force magnitudes, which are highly sensitive to simulation inaccuracies, force directions encode high-level task geometry and remain robust across the sim-to-real gap.
  arXiv  Detail & Related papers  (2026-02-15T14:57:13Z)
• Rigidity-Based Multi-Finger Coordination for Precise In-Hand Manipulation of Force-Sensitive Objects [5.313029497667778]
  Multi-fingered hands rely solely on fingertip point contacts and are not able to apply pull forces.<n>We propose a dual-layer framework for multi-finger coordination, enabling high-precision manipulation of force-sensitive objects.<n>We validate the framework on a custom dexterous hand, demonstrating the capability to manipulate fragile objects.
  arXiv  Detail & Related papers  (2026-02-15T11:40:39Z)
• Semantic-Contact Fields for Category-Level Generalizable Tactile Tool Manipulation [82.63833405368159]
  Generalizing tool manipulation requires both semantic planning and precise physical control.<n>We propose Semantic-Contact Fields (SCFields), a unified 3D representation fusing visual semantics with dense contact estimates.<n> Experiments on scraping, crayon drawing, and peeling demonstrate robust category-level generalization.
  arXiv  Detail & Related papers  (2026-02-14T16:05:08Z)
• Learning Force-Regulated Manipulation with a Low-Cost Tactile-Force-Controlled Gripper [18.40849472506029]
  TF-Gripper is a low-cost force-controlled parallel-jaw gripper that integrates tactile sensing as feedback.<n> RETAF is a framework that decouples grasping force control from arm pose prediction.<n>We evaluate TF-Gripper and RETAF across five real-world tasks requiring precise force regulation.
  arXiv  Detail & Related papers  (2026-02-10T17:36:33Z)
• UniForce: A Unified Latent Force Model for Robot Manipulation with Diverse Tactile Sensors [51.88112610411651]
  We propose a novel unified tactile representation learning framework that learns a shared latent force space across diverse tactile sensors.<n>UniForce reduces cross-sensor domain shift by jointly modeling inverse dynamics (image-to-force) and forward dynamics (force-to-image)<n>To avoid reliance on expensive external force/torque (F/T) sensors, we exploit static equilibrium and collect force-paired data via direct sensor--object--sensor interactions.
  arXiv  Detail & Related papers  (2026-02-01T11:03:01Z)
• Global End-Effector Pose Control of an Underactuated Aerial Manipulator via Reinforcement Learning [15.634803296068009]
  Aerial manipulators, which combine robotic arms with multi-rotor drones, face strict constraints on arm weight and mechanical complexity.<n>We study a lightweight 2-degree-of-freedom (DoF) arm mounted on a quadrotor via a differential mechanism, capable of full six-DoF end-effector pose control.
  arXiv  Detail & Related papers  (2025-12-24T10:00:01Z)
• ForceVLA: Enhancing VLA Models with a Force-aware MoE for Contact-rich Manipulation [62.58034332427291]
  ForceVLA is a novel end-to-end manipulation framework.<n>It treats external force sensing as a first-class modality within VLA systems.
  arXiv  Detail & Related papers  (2025-05-28T09:24:25Z)
• ForceGrip: Reference-Free Curriculum Learning for Realistic Grip Force Control in VR Hand Manipulation [0.10995326465245926]
  We present ForceGrip, a deep learning agent that synthesizes realistic hand manipulation motions.<n>We employ a three-phase curriculum learning framework comprising Finger Positioning, Intention Adaptation, and Dynamic Stabilization.<n>Our evaluations reveal ForceGrip's superior force controllability and plausibility compared to state-of-the-art methods.
  arXiv  Detail & Related papers  (2025-03-11T05:39:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.