KeyPointDiffuser: Unsupervised 3D Keypoint Learning via Latent Diffusion Models

• URL: http://arxiv.org/abs/2512.03450v1
• Date: Wed, 03 Dec 2025 05:08:03 GMT
• Title: KeyPointDiffuser: Unsupervised 3D Keypoint Learning via Latent Diffusion Models
• Authors: Rhys Newbury, Juyan Zhang, Tin Tran, Hanna Kurniawati, Dana Kulić,
• Abstract summary: We present an unsupervised framework for learning spatially structured 3D keypoints from point cloud data.<n>These keypoints serve as a compact and interpretable representation that conditions an Elucidated Diffusion Model (EDM) to reconstruct the full shape.<n>Our method achieves strong performance across diverse object categories, yielding a 6 percentage-point improvement in keypoint consistency compared to prior approaches.
• Score: 3.6047642906482147
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Understanding and representing the structure of 3D objects in an unsupervised manner remains a core challenge in computer vision and graphics. Most existing unsupervised keypoint methods are not designed for unconditional generative settings, restricting their use in modern 3D generative pipelines; our formulation explicitly bridges this gap. We present an unsupervised framework for learning spatially structured 3D keypoints from point cloud data. These keypoints serve as a compact and interpretable representation that conditions an Elucidated Diffusion Model (EDM) to reconstruct the full shape. The learned keypoints exhibit repeatable spatial structure across object instances and support smooth interpolation in keypoint space, indicating that they capture geometric variation. Our method achieves strong performance across diverse object categories, yielding a 6 percentage-point improvement in keypoint consistency compared to prior approaches.

Related papers

• Adaptive Point-Prompt Tuning: Fine-Tuning Heterogeneous Foundation Models for 3D Point Cloud Analysis [51.37795317716487]
  We propose the Adaptive Point-Prompt Tuning (APPT) method, which fine-tunes pre-trained models with a modest number of parameters.<n>We convert raw point clouds into point embeddings by aggregating local geometry to capture spatial features followed by linear layers.<n>To calibrate self-attention across source domains of any modality to 3D, we introduce a prompt generator that shares weights with the point embedding module.
  arXiv  Detail & Related papers  (2025-08-30T06:02:21Z)
• Pts3D-LLM: Studying the Impact of Token Structure for 3D Scene Understanding With Large Language Models [9.658828841170472]
  This work presents a rigorous study of 3D token structures, systematically comparing video-based and point-based representations.<n>We propose a novel approach that enriches visual tokens by incorporating 3D point cloud features from a Sonata pretrained Point Transformer V3 encoder.
  arXiv  Detail & Related papers  (2025-06-06T02:35:26Z)
• SelfGeo: Self-supervised and Geodesic-consistent Estimation of Keypoints on Deformable Shapes [19.730602733938216]
  "SelfGeo" is a self-supervised method that computes persistent 3D keypoints of non-rigid objects from arbitrary PCDs without the need of human annotations. Our main contribution is to enforce that keypoints deform along with the shape while keeping constant geodesic distances among them. We show experimentally that the use of geodesic has a clear advantage in challenging dynamic scenes.
  arXiv  Detail & Related papers  (2024-08-05T08:00:30Z)
• A Lightweight UDF Learning Framework for 3D Reconstruction Based on Local Shape Functions [42.840655419509346]
  This paper presents a novel neural framework, LoSF-UDF, for reconstructing surfaces from 3D point clouds by leveraging local shape functions.<n>Our method exhibits enhanced resilience to noise and outliers in point clouds compared to existing methods.
  arXiv  Detail & Related papers  (2024-07-01T14:39:03Z)
• Flattening-Net: Deep Regular 2D Representation for 3D Point Cloud Analysis [66.49788145564004]
  We present an unsupervised deep neural architecture called Flattening-Net to represent irregular 3D point clouds of arbitrary geometry and topology. Our methods perform favorably against the current state-of-the-art competitors.
  arXiv  Detail & Related papers  (2022-12-17T15:05:25Z)
• SNAKE: Shape-aware Neural 3D Keypoint Field [62.91169625183118]
  Detecting 3D keypoints from point clouds is important for shape reconstruction. This work investigates the dual question: can shape reconstruction benefit 3D keypoint detection? We propose a novel unsupervised paradigm named SNAKE, which is short for shape-aware neural 3D keypoint field.
  arXiv  Detail & Related papers  (2022-06-03T17:58:43Z)
• Unsupervised Learning of 3D Semantic Keypoints with Mutual Reconstruction [11.164069907549756]
  3D semantic keypoints are category-level semantic consistent points on 3D objects. We present an unsupervised method to generate consistent semantic keypoints from point clouds explicitly. To the best of our knowledge, the proposed method is the first to mine 3D semantic consistent keypoints from a mutual reconstruction view.
  arXiv  Detail & Related papers  (2022-03-19T01:49:21Z)
• Unsupervised Learning of Visual 3D Keypoints for Control [104.92063943162896]
  Learning sensorimotor control policies from high-dimensional images crucially relies on the quality of the underlying visual representations. We propose a framework to learn such a 3D geometric structure directly from images in an end-to-end unsupervised manner. These discovered 3D keypoints tend to meaningfully capture robot joints as well as object movements in a consistent manner across both time and 3D space.
  arXiv  Detail & Related papers  (2021-06-14T17:59:59Z)
• KeypointDeformer: Unsupervised 3D Keypoint Discovery for Shape Control [64.46042014759671]
  KeypointDeformer is an unsupervised method for shape control through automatically discovered 3D keypoints. Our approach produces intuitive and semantically consistent control of shape deformations.
  arXiv  Detail & Related papers  (2021-04-22T17:59:08Z)
• RfD-Net: Point Scene Understanding by Semantic Instance Reconstruction [19.535169371240073]
  We introduce RfD-Net that jointly detects and reconstructs dense object surfaces directly from point clouds. We decouple the instance reconstruction into global object localization and local shape prediction. Our approach consistently outperforms the state-of-the-arts and improves over 11 of mesh IoU in object reconstruction.
  arXiv  Detail & Related papers  (2020-11-30T12:58:05Z)
• CoKe: Localized Contrastive Learning for Robust Keypoint Detection [24.167397429511915]
  We show that keypoint kernels can be chosen to optimize three types of distances in the feature space. We formulate this optimization process within a framework, which includes supervised contrastive learning. CoKe achieves state-of-the-art results compared to approaches that jointly represent all keypoints holistically.
  arXiv  Detail & Related papers  (2020-09-29T16:00:43Z)
• Implicit Functions in Feature Space for 3D Shape Reconstruction and Completion [53.885984328273686]
  Implicit Feature Networks (IF-Nets) deliver continuous outputs, can handle multiple topologies, and complete shapes for missing or sparse input data. IF-Nets clearly outperform prior work in 3D object reconstruction in ShapeNet, and obtain significantly more accurate 3D human reconstructions.
  arXiv  Detail & Related papers  (2020-03-03T11:14:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.