Learning In-Hand Object Reaching to General 6D Poses

• URL: http://arxiv.org/abs/2609.13761v1
• Date: Sat, 12 Sep 2026 07:11:37 GMT
• Title: Learning In-Hand Object Reaching to General 6D Poses
• Abstract summary: In-hand manipulation allows multi-fingered dexterous hands to reconfigure grasped objects without releasing and regrasping them.<n>We formulate this capability as in-hand 6D object pose reaching: starting from an existing grasp, coordinated finger motions move the object to a palm-relative target pose.<n>We present POISE, a sim-to-real reinforcement learning framework for this task.
• Score: 20.489992320820324
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In-hand manipulation allows multi-fingered dexterous hands to reconfigure grasped objects without releasing and regrasping them. This improves manipulation efficiency by reducing repeated grasp acquisition and large arm motions. However, most learning-based methods focus on reorientation, continuous rotation, or translation, whereas many tasks require joint control of object position and orientation. We formulate this capability as in-hand 6D object pose reaching: starting from an existing grasp, coordinated finger motions move the object to a palm-relative target pose. We present POISE (Palm-relative Object reaching In SE(3)), a sim-to-real reinforcement learning framework for this task. POISE combines diverse stable-grasp initialization, goal- and geometry-conditioned control, an adaptive 6D goal curriculum, and a compact reward scheme for pose reaching and grasp preservation. In simulation, diverse initialization raises held-out-grasp success from 40.1% to 51.5% and post-drop recovery from 33.8% to 72.9%; the curriculum raises full-range success from 6.2% to 59.5%. On hardware, the grasp-maintenance reward improves three-target sequence success from 20% to 80%. In real-world experiments, POISE reaches successive 6D targets without manual reset across multiple object geometries and wrist orientations, and recovers from external disturbances. To support further research in dexterous manipulation, we will release our code at https://junxiaolin.github.io/poise-website/.

Related papers

• Cross-Embodiment Dexterous Hand Articulation Generation via Morphology-Aware Learning [82.63833405368159]
  Existing end-to-end methods require training on large-scale datasets for specific hands.<n>We propose an eigengrasp-based, end-to-end framework for cross-embodiment grasp generation.
  arXiv  Detail & Related papers  (2025-10-07T15:57:00Z)
• Camera Pose Refinement via 3D Gaussian Splatting [9.142303730601162]
  Camera pose refinement aims at improving the accuracy of initial pose estimation for applications in 3D computer vision.<n>We propose a novel camera pose refinement framework leveraging 3D Gaussian Splatting (3DGS)<n>Our method achieves 53.3% and 56.9% reductions in median translation and rotation errors on 7-Scenes and the Cambridge Landmarks datasets.
  arXiv  Detail & Related papers  (2025-08-25T10:29:59Z)
• Learning Point Cloud Representations with Pose Continuity for Depth-Based Category-Level 6D Object Pose Estimation [5.473857267110271]
  We propose HRC-Pose, a novel depth-only framework for category-level object pose estimation.<n>HRC-Pose decouples object pose into rotation and translation components, which are separately encoded and leveraged throughout the network.<n>Results on REAL275 and CAMERA25 benchmarks show that our method consistently outperforms existing depth-only state-of-the-art methods.
  arXiv  Detail & Related papers  (2025-08-20T02:09:02Z)
• DexCatch: Learning to Catch Arbitrary Objects with Dexterous Hands [14.712280514097912]
  We propose a Learning-based framework for Throwing-Catching tasks using dexterous hands. Our method achieves a 73% success rate across 45 scenarios (diverse hand poses and objects) In tasks where the object in hand faces sideways, an extremely unstable scenario due to the lack of support from the palm, our method still achieves a success rate of over 60%.
  arXiv  Detail & Related papers  (2023-10-13T01:36:46Z)
• Learning to Estimate 6DoF Pose from Limited Data: A Few-Shot, Generalizable Approach using RGB Images [60.0898989456276]
  We present a new framework named Cas6D for few-shot 6DoF pose estimation that is generalizable and uses only RGB images. To address the false positives of target object detection in the extreme few-shot setting, our framework utilizes a self-supervised pre-trained ViT to learn robust feature representations. Experimental results on the LINEMOD and GenMOP datasets demonstrate that Cas6D outperforms state-of-the-art methods by 9.2% and 3.8% accuracy (Proj-5) under the 32-shot setting.
  arXiv  Detail & Related papers  (2023-06-13T07:45:42Z)
• HACMan: Learning Hybrid Actor-Critic Maps for 6D Non-Prehensile Manipulation [29.01984677695523]
  We introduce Hybrid Actor-Critic Maps for Manipulation (HACMan), a reinforcement learning approach for 6D non-prehensile manipulation of objects. We evaluate HACMan on a 6D object pose alignment task in both simulation and in the real world. Compared to alternative action representations, HACMan achieves a success rate more than three times higher than the best baseline.
  arXiv  Detail & Related papers  (2023-05-06T05:55:27Z)
• Coupled Iterative Refinement for 6D Multi-Object Pose Estimation [64.7198752089041]
  Given a set of known 3D objects and an RGB or RGB-D input image, we detect and estimate the 6D pose of each object. Our approach iteratively refines both pose and correspondence in a tightly coupled manner, allowing us to dynamically remove outliers to improve accuracy.
  arXiv  Detail & Related papers  (2022-04-26T18:00:08Z)
• SO-Pose: Exploiting Self-Occlusion for Direct 6D Pose Estimation [98.83762558394345]
  SO-Pose is a framework for regressing all 6 degrees-of-freedom (6DoF) for the object pose in a cluttered environment from a single RGB image. We introduce a novel reasoning about self-occlusion, in order to establish a two-layer representation for 3D objects. Cross-layer consistencies that align correspondences, self-occlusion and 6D pose, we can further improve accuracy and robustness.
  arXiv  Detail & Related papers  (2021-08-18T19:49:29Z)
• Contact-GraspNet: Efficient 6-DoF Grasp Generation in Cluttered Scenes [50.303361537562715]
  We propose an end-to-end network that efficiently generates a distribution of 6-DoF parallel-jaw grasps. By rooting the full 6-DoF grasp pose and width in the observed point cloud, we can reduce the dimensionality of our grasp representation to 4-DoF. In a robotic grasping study of unseen objects in structured clutter we achieve over 90% success rate, cutting the failure rate in half compared to a recent state-of-the-art method.
  arXiv  Detail & Related papers  (2021-03-25T20:33:29Z)
• EfficientPose: An efficient, accurate and scalable end-to-end 6D multi object pose estimation approach [0.0]
  We introduce EfficientPose, a new approach for 6D object pose estimation. It is highly accurate, efficient and scalable over a wide range of computational resources. It can detect the 2D bounding box of multiple objects and instances as well as estimate their full 6D poses in a single shot.
  arXiv  Detail & Related papers  (2020-11-09T10:23:55Z)
• Goal-Auxiliary Actor-Critic for 6D Robotic Grasping with Point Clouds [62.013872787987054]
  We propose a new method for learning closed-loop control policies for 6D grasping. Our policy takes a segmented point cloud of an object from an egocentric camera as input, and outputs continuous 6D control actions of the robot gripper for grasping the object.
  arXiv  Detail & Related papers  (2020-10-02T07:42:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.