Rotation-Aware Point-Cloud Embeddings for Vision-Based In-Hand Reorientation

• URL: http://arxiv.org/abs/2606.21788v1
• Date: Fri, 19 Jun 2026 22:38:51 GMT
• Title: Rotation-Aware Point-Cloud Embeddings for Vision-Based In-Hand Reorientation
• Abstract summary: Point-cloud goals provide a way to specify dexterous in-hand reorientation.<n>Current and goal clouds are unordered, independently sampled, and often visibility-dependent.<n>We learn a rotation-aware point-cloud embedding whose Euclidean latent distance is calibrated to the SO(3) geodesic error between object orientations.
• Score: 7.551200858185237
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Point-cloud goals provide a direct way to specify dexterous in-hand reorientation: instead of defining an object-specific pose frame or estimating 6D pose at test time, the policy is given the desired 3D geometry of the object. Yet raw point-cloud goal conditioning is poorly conditioned for policy learning. Current and goal clouds are unordered, independently sampled, and often visibility-dependent, so their discrepancy entangles object rotation with permutation, resampling, and unstable correspondence structure. For this reason, prior point-cloud manipulation methods typically add structure outside the representation itself, such as explicit pose or relative-pose inputs, dense flow features, or distillation from privileged teachers. We close this gap by learning a rotation-aware point-cloud embedding whose Euclidean latent distance is calibrated to the SO(3) geodesic error between object orientations. The resulting representation turns current-goal comparison into a smooth control signal, allowing a model-free RL policy to act from current and goal point-cloud embeddings, proprioception, and centroid metadata, without object pose, relative pose, dense flow, or teacher-action supervision. In in-hand reorientation experiments, this interface matches privileged-state and distillation-based baselines while avoiding brittle test-time computation of structured pose or flow inputs. These results suggest that point-cloud goals become practical for this task when the representation, rather than an external module, encodes the task-relevant geometry of rotation. We also show evidence that generic visual point-cloud pretraining is insufficient for such a current-goal comparison because it discards the task-relevant state and preserves only shape features.

Related papers

• Adaptive Point-Prompt Tuning: Fine-Tuning Heterogeneous Foundation Models for 3D Point Cloud Analysis [51.37795317716487]
  We propose the Adaptive Point-Prompt Tuning (APPT) method, which fine-tunes pre-trained models with a modest number of parameters.<n>We convert raw point clouds into point embeddings by aggregating local geometry to capture spatial features followed by linear layers.<n>To calibrate self-attention across source domains of any modality to 3D, we introduce a prompt generator that shares weights with the point embedding module.
  arXiv  Detail & Related papers  (2025-08-30T06:02:21Z)
• Surfel-based 3D Registration with Equivariant SE(3) Features [34.796697445601914]
  Point cloud registration is crucial for ensuring 3D alignment consistency of multiple local point clouds in 3D reconstruction for remote sensing or digital heritage.<n>We propose a novel surfel-based pose learning regression approach to address these issues.<n>Our method can initialize surfels from Lidar point cloud using virtual perspective camera parameters, and learns explicit $mathbfSE(3)$ equivariant features.
  arXiv  Detail & Related papers  (2025-08-28T13:53:44Z)
• A Representation Separation Perspective to Correspondences-free Unsupervised 3D Point Cloud Registration [40.12490804387776]
  3D point cloud registration in remote sensing field has been greatly advanced by deep learning based methods. We propose a correspondences-free unsupervised point cloud registration (UPCR) method from the representation separation perspective. Our method not only filters out the disturbance in pose-invariant representation but also is robust to partial-to-partial point clouds or noise.
  arXiv  Detail & Related papers  (2022-03-24T17:50:19Z)
• IDEA-Net: Dynamic 3D Point Cloud Interpolation via Deep Embedding Alignment [58.8330387551499]
  We formulate the problem as estimation of point-wise trajectories (i.e., smooth curves) We propose IDEA-Net, an end-to-end deep learning framework, which disentangles the problem under the assistance of the explicitly learned temporal consistency. We demonstrate the effectiveness of our method on various point cloud sequences and observe large improvement over state-of-the-art methods both quantitatively and visually.
  arXiv  Detail & Related papers  (2022-03-22T10:14:08Z)
• Optimal Target Shape for LiDAR Pose Estimation [1.9048510647598205]
  Targets are essential in problems such as object tracking in cluttered or textureless environments. symmetric shapes lead to pose ambiguity when using sparse sensor data. This paper introduces the concept of optimizing target shape to remove pose ambiguity for LiDAR point clouds.
  arXiv  Detail & Related papers  (2021-09-02T19:18:24Z)
• SCTN: Sparse Convolution-Transformer Network for Scene Flow Estimation [71.2856098776959]
  Estimating 3D motions for point clouds is challenging, since a point cloud is unordered and its density is significantly non-uniform. We propose a novel architecture named Sparse Convolution-Transformer Network (SCTN) that equips the sparse convolution with the transformer. We show that the learned relation-based contextual information is rich and helpful for matching corresponding points, benefiting scene flow estimation.
  arXiv  Detail & Related papers  (2021-05-10T15:16:14Z)
• PRIN/SPRIN: On Extracting Point-wise Rotation Invariant Features [91.2054994193218]
  We propose a point-set learning framework PRIN, focusing on rotation invariant feature extraction in point clouds analysis. In addition, we extend PRIN to a sparse version called SPRIN, which directly operates on sparse point clouds. Results show that, on the dataset with randomly rotated point clouds, SPRIN demonstrates better performance than state-of-the-art methods without any data augmentation.
  arXiv  Detail & Related papers  (2021-02-24T06:44:09Z)
• 3D Object Classification on Partial Point Clouds: A Practical Perspective [91.81377258830703]
  A point cloud is a popular shape representation adopted in 3D object classification. This paper introduces a practical setting to classify partial point clouds of object instances under any poses. A novel algorithm in an alignment-classification manner is proposed in this paper.
  arXiv  Detail & Related papers  (2020-12-18T04:00:56Z)
• RfD-Net: Point Scene Understanding by Semantic Instance Reconstruction [19.535169371240073]
  We introduce RfD-Net that jointly detects and reconstructs dense object surfaces directly from point clouds. We decouple the instance reconstruction into global object localization and local shape prediction. Our approach consistently outperforms the state-of-the-arts and improves over 11 of mesh IoU in object reconstruction.
  arXiv  Detail & Related papers  (2020-11-30T12:58:05Z)
• Deep Positional and Relational Feature Learning for Rotation-Invariant Point Cloud Analysis [107.9979381402172]
  We propose a rotation-invariant deep network for point clouds analysis. The network is hierarchical and relies on two modules: a positional feature embedding block and a relational feature embedding block. Experiments show state-of-the-art classification and segmentation performances on benchmark datasets.
  arXiv  Detail & Related papers  (2020-11-18T04:16:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.