Topology-Agnostic Mesh Reconstruction of Deformable Objects from Sparse Touch

• URL: http://arxiv.org/abs/2607.13479v1
• Date: Wed, 15 Jul 2026 06:17:35 GMT
• Title: Topology-Agnostic Mesh Reconstruction of Deformable Objects from Sparse Touch
• Abstract summary: We present a single topology-agnostic estimator that reconstructs the full mesh of a deformable object from only a few touches and no vision.<n>We show that the estimator's deep-ensemble uncertainty can be used to learn where to touch next.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Estimating the full shape of a deformable object is especially challenging when vision is unavailable: in the dark, inside an opaque bag, behind the manipulating hand, or under heavy self-occlusion. Touch is the natural sensor in these settings, but touches are sparse and local. We present a single topology-agnostic estimator that reconstructs the full mesh of a deformable object from only a few touches and no vision, using one permutation-invariant cross-attention architecture that handles a 1D rope, a 2D cloth, and a 3D volumetric soft body. The learned estimator reduces reconstruction error by roughly two-thirds relative to non-learned geometric mesh completion and a Gaussian-process surface baseline, and it outperforms a simpler global-pool set encoder, with the gap growing as more touches are observed. We then show that the estimator's deep-ensemble uncertainty can be used to learn where to touch next, which lowers error further and beats both random touching and a Gaussian-process active baseline at sparse budgets. This gain is modest on average but grows with self-occlusion and on the error tail. When vision is also available, where to touch barely matters, motivating the vision-free setting we study.

Related papers

• Reliability-Aware 3D Geometric Injection for Universal Person Re-identification [47.874819213762926]
  We propose a Universal Monocular 3D-Enhanced ReID framework driven by a Consistency-Aware Reliability Gate and Dual-Stream Residual Fusion.<n>Our method improves challenging, structure-sensitive scenarios while preserving competitive performance on clean domains.
  arXiv  Detail & Related papers  (2026-07-21T08:54:25Z)
• Imagine2Real: Towards Zero-shot Humanoid-Object Interaction via Video Generative Priors [51.096845970243855]
  Whole-body Humanoid-Object Interaction (HOI) is bottlenecked by the scarcity of high-fidelity 3D data.<n>We propose Imagine2Real, a zero-shot HOI framework for flexible, geometry-free interaction.
  arXiv  Detail & Related papers  (2026-05-21T10:15:39Z)
• Learning 3D Representations for Spatial Intelligence from Unposed Multi-View Images [81.94999489820974]
  UniSplat is a feed-forward framework for learning 3D representations from unposed multi-view images.<n>We introduce a dual-masking strategy that strengthens geometry induction in the encoder.<n>Second, we develop a coarse-to-fine Gaussian splatting strategy that reduces appearance-semantics inconsistencies.<n>Third, we introduce a pose-conditioned recalibration mechanism that interrelates predicted 3D point and semantic maps into the image plane.
  arXiv  Detail & Related papers  (2026-04-12T10:36:18Z)
• TouchAnything: Diffusion-Guided 3D Reconstruction from Sparse Robot Touches [18.3533066960967]
  We present TouchAnything, a framework that leverages a pretrained vision diffusion model as a semantic and geometric prior for 3D reconstruction from sparse tactile measurements.<n>Our method reconstructs accurate geometries from only a few touches, outperforms existing baselines, and enables open-world 3D reconstruction of previously unseen object instances.
  arXiv  Detail & Related papers  (2026-04-10T04:26:36Z)
• SpatialActor: Exploring Disentangled Spatial Representations for Robust Robotic Manipulation [63.48859753472547]
  SpatialActor is a framework for robust robotic manipulation that explicitly decouples semantics and geometry.<n>It achieves state-of-the-art performance with 87.4% on RLBench and improves by 13.9% to 19.4% under varying noisy conditions.
  arXiv  Detail & Related papers  (2025-11-12T18:59:08Z)
• Parameterization-driven Neural Surface Reconstruction for Object-oriented Editing in Neural Rendering [35.69582529609475]
  This paper introduces a novel neural algorithm for parameterizing neural implicit surfaces to simple parametric domains like spheres and polycubes. It computes bi-directional deformation between the object and the domain using a forward mapping from the object's zero level set and an inverse deformation for backward mapping. We demonstrate the method's effectiveness on images of human heads and man-made objects.
  arXiv  Detail & Related papers  (2023-10-09T08:42:40Z)
• Decaf: Monocular Deformation Capture for Face and Hand Interactions [77.75726740605748]
  This paper introduces the first method that allows tracking human hands interacting with human faces in 3D from single monocular RGB videos. We model hands as articulated objects inducing non-rigid face deformations during an active interaction. Our method relies on a new hand-face motion and interaction capture dataset with realistic face deformations acquired with a markerless multi-view camera system.
  arXiv  Detail & Related papers  (2023-09-28T17:59:51Z)
• Sampling is Matter: Point-guided 3D Human Mesh Reconstruction [0.0]
  This paper presents a simple yet powerful method for 3D human mesh reconstruction from a single RGB image. Experimental results on benchmark datasets show that the proposed method efficiently improves the performance of 3D human mesh reconstruction.
  arXiv  Detail & Related papers  (2023-04-19T08:45:26Z)
• MonoJSG: Joint Semantic and Geometric Cost Volume for Monocular 3D Object Detection [10.377424252002792]
  monocular 3D object detection lacks accurate depth recovery ability. Deep neural network (DNN) enables monocular depth-sensing from high-level learned features. We propose a joint semantic and geometric cost volume to model the depth error.
  arXiv  Detail & Related papers  (2022-03-16T11:54:10Z)
• Cuboids Revisited: Learning Robust 3D Shape Fitting to Single RGB Images [44.223070672713455]
  In particular, man-made environments commonly consist of volumetric primitives such as cuboids or cylinders. Previous approaches directly estimate shape parameters from a 2D or 3D input, and are only able to reproduce simple objects. We propose a robust estimator for primitive fitting, which can meaningfully abstract real-world environments using cuboids.
  arXiv  Detail & Related papers  (2021-05-05T13:36:00Z)
• Learning Deformable Tetrahedral Meshes for 3D Reconstruction [78.0514377738632]
  3D shape representations that accommodate learning-based 3D reconstruction are an open problem in machine learning and computer graphics. Previous work on neural 3D reconstruction demonstrated benefits, but also limitations, of point cloud, voxel, surface mesh, and implicit function representations. We introduce Deformable Tetrahedral Meshes (DefTet) as a particular parameterization that utilizes volumetric tetrahedral meshes for the reconstruction problem.
  arXiv  Detail & Related papers  (2020-11-03T02:57:01Z)
• Reinforced Axial Refinement Network for Monocular 3D Object Detection [160.34246529816085]
  Monocular 3D object detection aims to extract the 3D position and properties of objects from a 2D input image. Conventional approaches sample 3D bounding boxes from the space and infer the relationship between the target object and each of them, however, the probability of effective samples is relatively small in the 3D space. We propose to start with an initial prediction and refine it gradually towards the ground truth, with only one 3d parameter changed in each step. This requires designing a policy which gets a reward after several steps, and thus we adopt reinforcement learning to optimize it.
  arXiv  Detail & Related papers  (2020-08-31T17:10:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.