SoftVTBench: A Safety-Aware Visuo-Tactile Benchmark for Physically Constrained Robotic Manipulation of Deformable Objects

• URL: http://arxiv.org/abs/2607.04234v1
• Date: Sun, 05 Jul 2026 11:12:29 GMT
• Title: SoftVTBench: A Safety-Aware Visuo-Tactile Benchmark for Physically Constrained Robotic Manipulation of Deformable Objects
• Abstract summary: We present SoftVTBench, a safety-aware visuo-tactile benchmark for deformable object manipulation.<n>Built in Isaac Sim with finite-element-simulated deformable objects, SoftVTBench provides multi-view RGB observations, tactile sensing with marker motion, proprioception, and language instructions.
• Score: 56.16187327142104
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Deformable object manipulation poses challenges beyond task completion: successful execution must also maintain safe physical interaction, holding the object stably without slip or drop while avoiding excessive deformation. However, existing manipulation benchmarks are predominantly success-oriented and rarely evaluate whether a policy remains physically safe throughout execution. We present SoftVTBench, a safety-aware visuo-tactile benchmark for physically constrained deformable object manipulation. Built in Isaac Sim with finite-element-simulated deformable objects, SoftVTBench provides multi-view RGB observations, RGB tactile sensing with marker motion, proprioception, and language instructions, and defines four matched task suites over object type (deformable vs. rigid) and variation axis (object vs. spatial). It separately reports Goal Success and Safety Success; the latter additionally requires no drop and peak deformation below a calibrated object-specific threshold, measured from policy-hidden privileged Finite Element Method (FEM) states. We implement pi0.5-based baselines under this protocol. Experiments show that success-only evaluation substantially overstates policy performance, as a large fraction of goal-completing rollouts still violate physical safety. Furthermore, incorporating tactile sensing improves Safety Success (e.g., from 21.4% to 35.6% on object-centric deformable tasks) and reduces object deformation during execution, while maintaining comparable Goal Success. SoftVTBench provides a reproducible benchmark for studying visuo-tactile deformable manipulation under physical interaction constraints.

Related papers

• SoftVTBench: A Deformation-Aware Visuo-Tactile Dataset and Benchmark for Deformable-Object Manipulation [56.16187327142104]
  We introduce SoftVTBench, a visuo-tactile dataset for physical-interaction-aware deformable-object manipulation.<n>It contains 4,000 expert demonstrations and more than 50 assets, including volumetric deformable objects and visually matched rigid twins.<n>At 20 Hz, each episode synchronizes multi-view RGB, dual-finger tactile RGB and marker motion, proprioception, language, and binary and continuous gripper actions.
  arXiv  Detail & Related papers  (2026-08-19T08:58:47Z)
• PhysV2A: Reachability-Gated and Semantic-Mask-Constrained Feasibility Completion for Video-to-Robot Manipulation [17.70834422785338]
  Video-based manipulation provides object-centric motion priors from human demonstrations, generated videos, or RGB-D observations.<n>PhysV2A is a framework for converting video-derived 6D object motion into robot-executable manipulation trajectories.
  arXiv  Detail & Related papers  (2026-07-10T12:43:00Z)
• Monocular Vision Based Control Framework for Grasping [8.44254908378373]
  We present a vision-based grasping framework that targets both soft and rigid objects within a single control pipeline.<n>The proposed system combines open-vocabulary object detection, image segmentation, boundary-aware point assignment, real-time point tracking, and monocular depth estimation.<n>Results demonstrate that the framework achieves stable grasping across both soft and rigid objects using visual feedback alone.
  arXiv  Detail & Related papers  (2026-07-08T20:13:17Z)
• PACT: Self-Evolving Physical Safety Alignment for Diffusion Policies in Embodied Manipulation [38.43792325467372]
  Existing approaches impose safety either prematurely during training or reactively via external guardrails at test time.<n>We propose Physical safety Alignment for Constrained Trajectories (PACT), a self-evolving post-training framework that projects pretrained diffusion policies onto constraint-feasible regions.<n>On simulated and real-world embodied manipulation benchmarks, PACT significantly reduces safety violations by 31.0% on average while improving task success by 30.7%.
  arXiv  Detail & Related papers  (2026-06-07T02:27:53Z)
• VLESA: Vision-Language Embodied Safety Agent for Human Activity Monitoring [60.53395558502203]
  Vision-Language Embodied Safety Agent (VLESA) monitors human activities from egocentric video.<n>VLESA addresses intent-dependent safety where identical actions can be safe or dangerous depending on context.<n>An intent-action prediction agent is proposed to jointly infer goals and predict future actions from video.
  arXiv  Detail & Related papers  (2026-06-02T17:42:17Z)
• SafeVLA-Bench: A Benchmark for the Success-Safety Gap in Vision-Language-Action Models [5.9288463407781995]
  Vision-language-action (VLA) benchmarks measure whether a policy completes a requested manipulation task.<n>We present SafeVLA-Bench, a post-hoc safety-evaluation framework for existing simulator-based VLA benchmarks.
  arXiv  Detail & Related papers  (2026-05-30T15:31:57Z)
• SafeManip: A Property-Driven Benchmark for Temporal Safety Evaluation in Robotic Manipulation [30.582295885898997]
  We introduce SafeManip, a property-driven benchmark to explicitly evaluate temporal safety properties in robotic manipulation.<n>SafeManip defines reusable safety templates over finite executions using Linear Temporal Logic over finite traces (LTLf)<n>We evaluate SafeManip on six vision-language-action policies, including $_0.5$, GR00T, and their training variants, across 50 RoboCasa365 household tasks.
  arXiv  Detail & Related papers  (2026-05-12T16:49:28Z)
• GAPG: Geometry Aware Push-Grasping Synergy for Goal-Oriented Manipulation in Clutter [27.71481975135778]
  In cluttered environments, a single-step grasp is often insufficient.<n>Previous work has introduced pushing as an auxiliary action to create graspable space.<n>We propose a geometry-aware push-grasp synergy framework to integrate grasp and push evaluation.
  arXiv  Detail & Related papers  (2026-03-22T12:30:44Z)
• Dynamic Manipulation of Deformable Objects in 3D: Simulation, Benchmark and Learning Strategy [88.8665000676562]
  Prior methods often simplify the problem to low-speed or 2D settings, limiting their applicability to real-world 3D tasks.<n>To mitigate data scarcity, we introduce a novel simulation framework and benchmark grounded in reduced-order dynamics.<n>We propose Dynamics Informed Diffusion Policy (DIDP), a framework that integrates imitation pretraining with physics-informed test-time adaptation.
  arXiv  Detail & Related papers  (2025-05-23T03:28:25Z)
• DeepSimHO: Stable Pose Estimation for Hand-Object Interaction via Physics Simulation [81.11585774044848]
  We present DeepSimHO, a novel deep-learning pipeline that combines forward physics simulation and backward gradient approximation with a neural network. Our method noticeably improves the stability of the estimation and achieves superior efficiency over test-time optimization.
  arXiv  Detail & Related papers  (2023-10-11T05:34:36Z)
• A Bayesian Treatment of Real-to-Sim for Deformable Object Manipulation [59.29922697476789]
  We propose a novel methodology for extracting state information from image sequences via a technique to represent the state of a deformable object as a distribution embedding. Our experiments confirm that we can estimate posterior distributions of physical properties, such as elasticity, friction and scale of highly deformable objects, such as cloth and ropes.
  arXiv  Detail & Related papers  (2021-12-09T17:50:54Z)
• Tracking Partially-Occluded Deformable Objects while Enforcing Geometric Constraints [11.247580943940916]
  We build high-fidelity physics simulation to aid in tracking a deformable object. We focus on the object based on RGBD images and geometric motion estimates. Our contributions allow us to outperform previous methods by a large margin in terms of accuracy.
  arXiv  Detail & Related papers  (2020-11-01T21:13:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.