Bayesian Active Object Recognition and 6D Pose Estimation from Multimodal Contact Sensing

• URL: http://arxiv.org/abs/2603.21410v1
• Date: Sun, 22 Mar 2026 21:30:23 GMT
• Title: Bayesian Active Object Recognition and 6D Pose Estimation from Multimodal Contact Sensing
• Abstract summary: We present an active tactile exploration framework for joint object recognition and 6D pose estimation.<n>The proposed method integrates wrist force/torque sensing, GelSight tactile sensing, and free-space constraints.<n>Results show that incorporating tactile and free-space information substantially improves recognition and pose estimation accuracy and stability.
• Score: 14.370567509757555
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present an active tactile exploration framework for joint object recognition and 6D pose estimation. The proposed method integrates wrist force/torque sensing, GelSight tactile sensing, and free-space constraints within a Bayesian inference framework that maintains a belief over object class and pose during active tactile exploration. By combining contact and non-contact evidence, the framework reduces ambiguity and improves robustness in the joint class-pose estimation problem. To enable efficient inference in the large hypothesis space, we employ a customized particle filter that progressively samples particles based on new observations. The inferred belief is further used to guide active exploration by selecting informative next touches under reachability constraints. For effective data collection, a motion planning and control framework is developed to plan and execute feasible paths for tactile exploration, handle unexpected contacts and GelSight-surface alignment with tactile servoing. We evaluate the framework in simulation and on a Franka Panda robot using 11 YCB objects. Results show that incorporating tactile and free-space information substantially improves recognition and pose estimation accuracy and stability, while reducing the number of action cycles compared with force/torque-only baselines. Code, dataset, and supplementary material will be made available online.

Related papers

• Robust Slip Detection and Material Classification via Spatiotemporal Transformers on a Uniformly-Illuminated Visuo-Tactile Sensor [52.33204893988409]
  We propose a visuo-tactile sensor featuring customized uniform illumination, alongside a unified perception framework.<n>The proposed hardware-software framework provides high-fidelity feedback and a powerful multi-modal perception baseline for complex robotic manipulation.
  arXiv  Detail & Related papers  (2026-08-25T07:25:27Z)
• SpaceEra++: A Unified Framework Towards 3D Spatial Reasoning in Video [88.79625979053873]
  We propose a novel framework, SpaceEra, for pre-trained vision-language models (VLMs)<n>We extend the original framework into a comprehensive system, termed SpaceEra++, which spans data construction, model design, training optimization, and prompting inference.<n>In addition, to enhance spatial reasoning, we develop SpaceAlign, which enforces pairwise object constraints by jointly exploiting absolute coordinates and relative spatial relations.
  arXiv  Detail & Related papers  (2026-07-02T06:56:29Z)
• GAPG: Geometry Aware Push-Grasping Synergy for Goal-Oriented Manipulation in Clutter [27.71481975135778]
  In cluttered environments, a single-step grasp is often insufficient.<n>Previous work has introduced pushing as an auxiliary action to create graspable space.<n>We propose a geometry-aware push-grasp synergy framework to integrate grasp and push evaluation.
  arXiv  Detail & Related papers  (2026-03-22T12:30:44Z)
• Diffusion-based Inverse Model of a Distributed Tactile Sensor for Object Pose Estimation [45.95486121114164]
  We learn an inverse tactile sensor model using denoising diffusion to efficiently leverage tactile data for pose estimation.<n>The model is conditioned on tactile observations from a distributed tactile sensor and trained in simulation using a geometric sensor model.<n>For online pose estimation, we integrate the inverse model with a particle filter through a proposal scheme.
  arXiv  Detail & Related papers  (2026-01-19T17:36:28Z)
• Articulated Object Manipulation using Online Axis Estimation with SAM2-Based Tracking [57.942404069484134]
  Articulated object manipulation requires precise object interaction, where the object's axis must be carefully considered.<n>Previous research employed interactive perception for manipulating articulated objects, but typically, open-loop approaches often suffer from overlooking the interaction dynamics.<n>We present a closed-loop pipeline integrating interactive perception with online axis estimation from segmented 3D point clouds.
  arXiv  Detail & Related papers  (2024-09-24T17:59:56Z)
• A Bayesian Framework for Active Tactile Object Recognition, Pose Estimation and Shape Transfer Learning [3.35643050085202]
  In this paper, we address the problem of active tactile object recognition, pose estimation and shape transfer learning. A customized particle filter (PF) and a Gaussian process implicit surface (GPIS) are combined in a unified Bayesian framework. Through experiments in simulation, the proposed framework demonstrated its effectiveness and efficiency in estimating object class and pose for known objects and learning novel shapes.
  arXiv  Detail & Related papers  (2024-09-10T23:35:30Z)
• Towards Consistent Object Detection via LiDAR-Camera Synergy [17.665362927472973]
  There is no existing model capable of detecting an object's position in both point clouds and images. This paper introduces an end-to-end Consistency Object Detection (COD) algorithm framework. To assess the accuracy of the object correlation between point clouds and images, this paper proposes a new evaluation metric, Consistency Precision.
  arXiv  Detail & Related papers  (2024-05-02T13:04:26Z)
• Tactile Estimation of Extrinsic Contact Patch for Stable Placement [64.06243248525823]
  We present the design of feedback skills for robots that must learn to stack complex-shaped objects on top of each other. We estimate the contact patch between a grasped object and its environment using force and tactile observations.
  arXiv  Detail & Related papers  (2023-09-25T21:51:48Z)
• Tactile-Filter: Interactive Tactile Perception for Part Mating [54.46221808805662]
  Humans rely on touch and tactile sensing for a lot of dexterous manipulation tasks. vision-based tactile sensors are being widely used for various robotic perception and control tasks. We present a method for interactive perception using vision-based tactile sensors for a part mating task.
  arXiv  Detail & Related papers  (2023-03-10T16:27:37Z)
• TRiPOD: Human Trajectory and Pose Dynamics Forecasting in the Wild [77.59069361196404]
  TRiPOD is a novel method for predicting body dynamics based on graph attentional networks. To incorporate a real-world challenge, we learn an indicator representing whether an estimated body joint is visible/invisible at each frame. Our evaluation shows that TRiPOD outperforms all prior work and state-of-the-art specifically designed for each of the trajectory and pose forecasting tasks.
  arXiv  Detail & Related papers  (2021-04-08T20:01:00Z)
• Robust Ego and Object 6-DoF Motion Estimation and Tracking [5.162070820801102]
  This paper proposes a robust solution to achieve accurate estimation and consistent track-ability for dynamic multi-body visual odometry. A compact and effective framework is proposed leveraging recent advances in semantic instance-level segmentation and accurate optical flow estimation. A novel formulation, jointly optimizing SE(3) motion and optical flow is introduced that improves the quality of the tracked points and the motion estimation accuracy.
  arXiv  Detail & Related papers  (2020-07-28T05:12:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.