Zero Shot Deformation Reconstruction for Soft Robots Using a Flexible Sensor Array and Cage Based 3D Gaussian Modeling

• URL: http://arxiv.org/abs/2603.19543v1
• Date: Fri, 20 Mar 2026 00:56:39 GMT
• Title: Zero Shot Deformation Reconstruction for Soft Robots Using a Flexible Sensor Array and Cage Based 3D Gaussian Modeling
• Abstract summary: zero-shot deformation reconstruction is defined as the ability to infer object-wide deformations on unseen soft robots without collecting deformation data or performing any retraining during deployment.<n>The proposed framework integrates a flexible piezoresistive sensor array with a geometry-aware, cage-based 3D Gaussian deformation model.<n>It achieves 0.67 IoU, 0.65 SSIM, and 3.48 mm Chamfer distance, demonstrating strong zero-shot generalization through explicit coupling of tactile sensing and structured geometric deformation.
• Score: 13.363052557818968
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present a zero-shot deformation reconstruction framework for soft robots that operates without any visual supervision at inference time. In this work, zero-shot deformation reconstruction is defined as the ability to infer object-wide deformations on previously unseen soft robots without collecting object-specific deformation data or performing any retraining during deployment. Our method assumes access to a static geometric proxy of the undeformed object, which can be obtained from a STL model. During operation, the system relies exclusively on tactile sensing, enabling camera-free deformation inference. The proposed framework integrates a flexible piezoresistive sensor array with a geometry-aware, cage-based 3D Gaussian deformation model. Local tactile measurements are mapped to low-dimensional cage control signals and propagated to dense Gaussian primitives to generate globally consistent shape deformations. A graph attention network regresses cage displacements from tactile input, enforcing spatial smoothness and structural continuity via boundary-aware propagation. Given only a nominal geometric proxy and real-time tactile signals, the system performs zero-shot deformation reconstruction of unseen soft robots in bending and twisting motions, while rendering photorealistic RGB in real time. It achieves 0.67 IoU, 0.65 SSIM, and 3.48 mm Chamfer distance, demonstrating strong zero-shot generalization through explicit coupling of tactile sensing and structured geometric deformation.

Related papers

• OmniPoint: Universal Monocular Metric Pointcloud from Any Camera [79.24417597712802]
  We present OmniPoint, a unified framework designed to generalize metric reconstruction across diverse imaging sensors.<n>To overcome projection rigidity, OmniPoint abandons conventional planar depth regression.<n>It instead adopts a decoupled ray and distance representation alongside a decoupled training objective.
  arXiv  Detail & Related papers  (2026-09-08T19:45:58Z)
• GrainGS: Gradient-Decoupled Gaussian Splatting for Efficient Dynamic Novel View Synthesis [94.20462261657525]
  GrainGS is a dynamic framework that combines a hierarchical anchor scaffold with per-Gaussian deformation.<n>It achieves high reconstruction quality, real-time novel view synthesis, and compact storage.
  arXiv  Detail & Related papers  (2026-07-23T15:52:35Z)
• PhysiFormer: Learning to Simulate Mechanics in World Space [56.16644631495]
  We present PhysiFormer, a diffusion transformer for physically-plausible 3D object motion.<n>It represents objects as 3D meshes expressed in world coordinates.<n>It substantially outperforms autoregressive baselines in trajectory accuracy, rigidity preservation, and momentum-based physical consistency.
  arXiv  Detail & Related papers  (2026-06-25T17:59:00Z)
• Fishbone: From One 3D Asset to a Million Controllable Edits [26.573828978137104]
  Fishbone is a unified rib-spine representation for general shapes that supports controllable parametric mesh, reduced-space dynamics, and animation.<n> ribs control local profiles such as thickness, orientation, and cross-sectional variation, while the spine controls global bending, twisting, and stretching.<n>We further construct Fishbone-136K by augmenting Hunyuan3D with rib-spine structures, and demonstrate applications in controllable 3D generation, deformation-based data augmentation for robot learning, interactive mesh editing, and agentic generation.
  arXiv  Detail & Related papers  (2026-05-24T01:38:43Z)
• A skin-like conformal sensor for real-time shape mapping [2.128937994248034]
  Real-time 3D shape sensing is essential for robust control and interpretation of deformable systems during motion.<n>Here, we introduce a scalable, skin-like sensor that reconstructs its continuous 3D deformation in real time from distributed strain measurements.<n>A 5-by-5 array with a 12 mm pitch achieves a mean surface reconstruction error of 0.62 mm with 0.1s latency across all tested scenarios.
  arXiv  Detail & Related papers  (2026-05-02T00:19:19Z)
• A Model-based Visual Contact Localization and Force Sensing System for Compliant Robotic Grippers [11.232660351404684]
  Integrating force sensing into deformable grippers negotiates trade-offs in cost, complexity, mechanical robustness, and performance.<n>Current approaches mostly utilize end-to-end deep learning, which can be brittle when generalizing to new scenarios.<n>We developed a model-based visual force sensing approach integrating an iterative contact localization with generalization to unseen objects.
  arXiv  Detail & Related papers  (2026-05-01T00:34:19Z)
• TouchAnything: Diffusion-Guided 3D Reconstruction from Sparse Robot Touches [18.3533066960967]
  We present TouchAnything, a framework that leverages a pretrained vision diffusion model as a semantic and geometric prior for 3D reconstruction from sparse tactile measurements.<n>Our method reconstructs accurate geometries from only a few touches, outperforms existing baselines, and enables open-world 3D reconstruction of previously unseen object instances.
  arXiv  Detail & Related papers  (2026-04-10T04:26:36Z)
• Adaptive Scaling with Geometric and Visual Continuity of completed 3D objects [0.0]
  We introduce a part-aware scaling framework that transforms static completed SDFs into structurally coherent objects.<n>Our method overcomes the inherent rigidity of completed SDFs and is visually more appealing than global and naive selective scaling.
  arXiv  Detail & Related papers  (2026-02-13T13:05:54Z)
• Robo3R: Enhancing Robotic Manipulation with Accurate Feed-Forward 3D Reconstruction [70.06600045165905]
  3D spatial perception is fundamental to general robotic manipulation, yet obtaining reliable, high-quality 3D geometry remains challenging.<n>We introduce Robo3R, a manipulation-ready 3D reconstruction model that predicts accurate, metric-scale scene geometry directly from RGB images and robot states in real time.<n>We observe consistent gains in performance, suggesting the promise of this alternative 3D sensing module for robotic manipulation.
  arXiv  Detail & Related papers  (2026-02-10T18:58:15Z)
• DiffeoMorph: Learning to Morph 3D Shapes Using Differentiable Agent-Based Simulations [2.174820084855635]
  DiffeoMorph is an end-to-end differentiable framework for learning a morphogenesis protocol.<n>It guides a population of agents to morph into a target 3D shape.<n>We show that DiffeoMorph can form a range of shapes using only minimal cues.
  arXiv  Detail & Related papers  (2025-12-18T23:50:42Z)
• 3DPCNet: Pose Canonicalization for Robust Viewpoint-Invariant 3D Kinematic Analysis from Monocular RGB cameras [7.906702226082628]
  3DPCNet is a compact, estimator-agnostic module that operates directly on 3D joint coordinates.<n>Our method produces acceleration signals from video that show strong visual correspondence to ground-truth IMU sensor data.
  arXiv  Detail & Related papers  (2025-09-27T18:55:21Z)
• Adaptive Point-Prompt Tuning: Fine-Tuning Heterogeneous Foundation Models for 3D Point Cloud Analysis [51.37795317716487]
  We propose the Adaptive Point-Prompt Tuning (APPT) method, which fine-tunes pre-trained models with a modest number of parameters.<n>We convert raw point clouds into point embeddings by aggregating local geometry to capture spatial features followed by linear layers.<n>To calibrate self-attention across source domains of any modality to 3D, we introduce a prompt generator that shares weights with the point embedding module.
  arXiv  Detail & Related papers  (2025-08-30T06:02:21Z)
• Self-Supervised Multi-Part Articulated Objects Modeling via Deformable Gaussian Splatting and Progressive Primitive Segmentation [23.18517560629462]
  We introduce DeGSS, a unified framework that encodes articulated objects as deformable 3D Gaussian fields, embedding geometry, appearance, and motion in one compact representation.<n>To evaluate generalization and realism, we enlarge the synthetic PartNet-Mobility benchmark and release RS-Art, a real-to-sim dataset that pairs RGB captures with accurately reverse-engineered 3D models.
  arXiv  Detail & Related papers  (2025-06-11T12:32:16Z)
• Simplicits: Mesh-Free, Geometry-Agnostic, Elastic Simulation [18.45850302604534]
  We present a data-, mesh-, and grid-free solution for elastic simulation for any object in any geometric representation. For each object, we fit a small implicit neural network encoding varying weights that act as a reduced deformation basis. Our experiments demonstrate the versatility, accuracy, and speed of this approach on data including signed distance functions, point clouds, neural primitives, tomography scans, radiance fields, Gaussian splats, surface meshes, and volume meshes.
  arXiv  Detail & Related papers  (2024-06-09T18:57:23Z)
• Learning Modulated Transformation in GANs [69.95217723100413]
  We equip the generator in generative adversarial networks (GANs) with a plug-and-play module, termed as modulated transformation module (MTM) MTM predicts spatial offsets under the control of latent codes, based on which the convolution operation can be applied at variable locations. It is noteworthy that towards human generation on the challenging TaiChi dataset, we improve the FID of StyleGAN3 from 21.36 to 13.60, demonstrating the efficacy of learning modulated geometry transformation.
  arXiv  Detail & Related papers  (2023-08-29T17:51:22Z)
• SceNeRFlow: Time-Consistent Reconstruction of General Dynamic Scenes [75.9110646062442]
  We propose SceNeRFlow to reconstruct a general, non-rigid scene in a time-consistent manner. Our method takes multi-view RGB videos and background images from static cameras with known camera parameters as input. We show experimentally that, unlike prior work that only handles small motion, our method enables the reconstruction of studio-scale motions.
  arXiv  Detail & Related papers  (2023-08-16T09:50:35Z)
• Elastic Tactile Simulation Towards Tactile-Visual Perception [58.44106915440858]
  We propose Elastic Interaction of Particles (EIP) for tactile simulation. EIP models the tactile sensor as a group of coordinated particles, and the elastic property is applied to regulate the deformation of particles during contact. We further propose a tactile-visual perception network that enables information fusion between tactile data and visual images.
  arXiv  Detail & Related papers  (2021-08-11T03:49:59Z)
• Dense Non-Rigid Structure from Motion: A Manifold Viewpoint [162.88686222340962]
  Non-Rigid Structure-from-Motion (NRSfM) problem aims to recover 3D geometry of a deforming object from its 2D feature correspondences across multiple frames. We show that our approach significantly improves accuracy, scalability, and robustness against noise.
  arXiv  Detail & Related papers  (2020-06-15T09:15:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.