ArtReg: Visuo-Tactile based Pose Tracking and Manipulation of Unseen Articulated Objects

• URL: http://arxiv.org/abs/2511.06378v1
• Date: Sun, 09 Nov 2025 13:30:51 GMT
• Title: ArtReg: Visuo-Tactile based Pose Tracking and Manipulation of Unseen Articulated Objects
• Authors: Prajval Kumar Murali, Mohsen Kaboli,
• Abstract summary: We present a novel method for visuo-tactile-based tracking of unseen objects.<n>Our approach integrates visuo-tactile point clouds in an unscented Kalman Filter formulation.<n>We have extensively evaluated our approach on various types of unknown objects through real robot experiments.
• Score: 2.9793019246605676
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Robots operating in real-world environments frequently encounter unknown objects with complex structures and articulated components, such as doors, drawers, cabinets, and tools. The ability to perceive, track, and manipulate these objects without prior knowledge of their geometry or kinematic properties remains a fundamental challenge in robotics. In this work, we present a novel method for visuo-tactile-based tracking of unseen objects (single, multiple, or articulated) during robotic interaction without assuming any prior knowledge regarding object shape or dynamics. Our novel pose tracking approach termed ArtReg (stands for Articulated Registration) integrates visuo-tactile point clouds in an unscented Kalman Filter formulation in the SE(3) Lie Group for point cloud registration. ArtReg is used to detect possible articulated joints in objects using purposeful manipulation maneuvers such as pushing or hold-pulling with a two-robot team. Furthermore, we leverage ArtReg to develop a closed-loop controller for goal-driven manipulation of articulated objects to move the object into the desired pose configuration. We have extensively evaluated our approach on various types of unknown objects through real robot experiments. We also demonstrate the robustness of our method by evaluating objects with varying center of mass, low-light conditions, and with challenging visual backgrounds. Furthermore, we benchmarked our approach on a standard dataset of articulated objects and demonstrated improved performance in terms of pose accuracy compared to state-of-the-art methods. Our experiments indicate that robust and accurate pose tracking leveraging visuo-tactile information enables robots to perceive and interact with unseen complex articulated objects (with revolute or prismatic joints).

Related papers

• Articulated 3D Scene Graphs for Open-World Mobile Manipulation [55.97942733699124]
  We present MoMa-SG, a framework for building semantic-kinematic 3D scene graphs of articulated scenes.<n>We estimate articulation models using a novel unified twist estimation formulation.<n>We also introduce the novel Arti4D-Semantic dataset.
  arXiv  Detail & Related papers  (2026-02-18T10:40:35Z)
• Object Pose Estimation through Dexterous Touch [27.99244228962149]
  Our approach uses sensorimotor exploration to actively control a robot hand to interact with the object.<n>We show that our method can actively explore an object's surface to identify critical pose features without prior knowledge of the object's geometry.
  arXiv  Detail & Related papers  (2025-09-16T23:25:05Z)
• ScrewSplat: An End-to-End Method for Articulated Object Recognition [11.498029485126045]
  ScrewSplat is a simple end-to-end method that operates solely on RGB observations.<n>We demonstrate that our method achieves state-of-the-art recognition accuracy across a diverse set of articulated objects.
  arXiv  Detail & Related papers  (2025-08-04T07:45:31Z)
• Learning Video Generation for Robotic Manipulation with Collaborative Trajectory Control [72.00655365269]
  We present RoboMaster, a novel framework that models inter-object dynamics through a collaborative trajectory formulation.<n>Unlike prior methods that decompose objects, our core is to decompose the interaction process into three sub-stages: pre-interaction, interaction, and post-interaction.<n>Our method outperforms existing approaches, establishing new state-of-the-art performance in trajectory-controlled video generation for robotic manipulation.
  arXiv  Detail & Related papers  (2025-06-02T17:57:06Z)
• Object and Contact Point Tracking in Demonstrations Using 3D Gaussian Splatting [17.03927416536173]
  This paper introduces a method to enhance Interactive Imitation Learning (IIL) by extracting touch interaction points and tracking object movement from video demonstrations. The approach extends current IIL systems by providing robots with detailed knowledge of both where and how to interact with objects, particularly complex articulated ones like doors and drawers.
  arXiv  Detail & Related papers  (2024-11-05T23:28:57Z)
• Learning Object Properties Using Robot Proprioception via Differentiable Robot-Object Interaction [52.12746368727368]
  Differentiable simulation has become a powerful tool for system identification.<n>Our approach calibrates object properties by using information from the robot, without relying on data from the object itself.<n>We demonstrate the effectiveness of our method on a low-cost robotic platform.
  arXiv  Detail & Related papers  (2024-10-04T20:48:38Z)
• Articulated Object Manipulation using Online Axis Estimation with SAM2-Based Tracking [57.942404069484134]
  Articulated object manipulation requires precise object interaction, where the object's axis must be carefully considered.<n>Previous research employed interactive perception for manipulating articulated objects, but typically, open-loop approaches often suffer from overlooking the interaction dynamics.<n>We present a closed-loop pipeline integrating interactive perception with online axis estimation from segmented 3D point clouds.
  arXiv  Detail & Related papers  (2024-09-24T17:59:56Z)
• 3D Foundation Models Enable Simultaneous Geometry and Pose Estimation of Grasped Objects [13.58353565350936]
  We contribute methodology to jointly estimate the geometry and pose of objects grasped by a robot. Our method transforms the estimated geometry into the robot's coordinate frame. We empirically evaluate our approach on a robot manipulator holding a diverse set of real-world objects.
  arXiv  Detail & Related papers  (2024-07-14T21:02:55Z)
• RPMArt: Towards Robust Perception and Manipulation for Articulated Objects [56.73978941406907]
  We propose a framework towards Robust Perception and Manipulation for Articulated Objects ( RPMArt) RPMArt learns to estimate the articulation parameters and manipulate the articulation part from the noisy point cloud. We introduce an articulation-aware classification scheme to enhance its ability for sim-to-real transfer.
  arXiv  Detail & Related papers  (2024-03-24T05:55:39Z)
• InterTracker: Discovering and Tracking General Objects Interacting with Hands in the Wild [40.489171608114574]
  Existing methods rely on frame-based detectors to locate interacting objects. We propose to leverage hand-object interaction to track interactive objects. Our proposed method outperforms the state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-08-06T09:09:17Z)
• V-MAO: Generative Modeling for Multi-Arm Manipulation of Articulated Objects [51.79035249464852]
  We present a framework for learning multi-arm manipulation of articulated objects. Our framework includes a variational generative model that learns contact point distribution over object rigid parts for each robot arm.
  arXiv  Detail & Related papers  (2021-11-07T02:31:09Z)
• Object Detection and Pose Estimation from RGB and Depth Data for Real-time, Adaptive Robotic Grasping [0.0]
  We propose a system that performs real-time object detection and pose estimation, for the purpose of dynamic robot grasping. The proposed approach allows the robot to detect the object identity and its actual pose, and then adapt a canonical grasp in order to be used with the new pose. For training, the system defines a canonical grasp by capturing the relative pose of an object with respect to the gripper attached to the robot's wrist. During testing, once a new pose is detected, a canonical grasp for the object is identified and then dynamically adapted by adjusting the robot arm's joint angles.
  arXiv  Detail & Related papers  (2021-01-18T22:22:47Z)
• "What's This?" -- Learning to Segment Unknown Objects from Manipulation Sequences [27.915309216800125]
  We present a novel framework for self-supervised grasped object segmentation with a robotic manipulator. We propose a single, end-to-end trainable architecture which jointly incorporates motion cues and semantic knowledge. Our method neither depends on any visual registration of a kinematic robot or 3D object models, nor on precise hand-eye calibration or any additional sensor data.
  arXiv  Detail & Related papers  (2020-11-06T10:55:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.