SR-JEPA: Learning Predictive Latent State in 3D Scenes

• URL: http://arxiv.org/abs/2608.05774v1
• Date: Thu, 06 Aug 2026 09:09:16 GMT
• Title: SR-JEPA: Learning Predictive Latent State in 3D Scenes
• Abstract summary: SR-JEPA is a point-native JEPA for scene-scale point clouds.<n>Training uses only self-contained 3D EMA targets.<n>On 5,953 held-out ARKitScenes objects, the imputed latent reaches 43.13% semantic-identity macro accuracy.
• Score: 8.534434711262909
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Joint-embedding predictive architectures learn by predicting latent representations of missing observations, yet many masked JEPAs are evaluated primarily through the encoders they produce. We ask what a trained predictive pathway itself infers when an entire entity is absent from a native 3D scene. We introduce SR-JEPA, a point-native JEPA for scene-scale point clouds whose original frozen predictive pathway can be queried at a supplied location. At evaluation, every point of one object is removed before encoding and replaced by the same shape-free 32-point query at its centroid. Training uses only self-contained 3D EMA targets: no reconstruction, semantic labels, language, or lifted 2D features. On 5,953 held-out ARKitScenes objects, the imputed latent reaches 43.13% semantic-identity macro accuracy, 22.18 points above the strongest floor. Randomizing the prediction path removes 9.78 points, while substituting matched donor context removes 21.98 points. On 8,570 Sr3D support pairs, the full latent reaches 41.15 AP; identity decoded from the missing-object latent, combined with anchor identity and geometry, reaches 39.37 AP, leaving an unresolved 1.78-point residual. These results reveal a queryable, compositional 3D predictive state: the model completes context-dependent entity content, which downstream computation combines with metric geometry.

Related papers

• P-POSEMEM: Projective Semantic Memory for Consistent Language Grounding under Pose-Graph Rewrites [10.214485367791779]
  P-POSEMEM stores each observation as an immutable event at its birth.<n>Dproj measures the total-variation defect between the language-goal distributions of inference-equivalent full and marginalized graphs.
  arXiv  Detail & Related papers  (2026-09-14T12:31:38Z)
• Efficient Visual Pointing for Embodied AI:Agent-Driven Data Synthesis, Cross-Block Attention, and Iterative Correction [55.11480729304395]
  We describe our PointArena 2026 solution, which achieves 77.2% overall accuracy and ranks second on the benchmark.<n>The ap proach targets three failure modes. First, agent-driven syn thesis builds large semantic and anchor-relative candidate pools.<n>Second, a determinis tic steerable-data pipeline creates a verified 10,000-sample main set, plus reserve samples, using masks, templates, and path verification.
  arXiv  Detail & Related papers  (2026-06-29T06:39:03Z)
• Rethinking Point Clouds as Sequences: A Causal Next-Token Predictive Learning Framework [31.937606388374885]
  We introduce PointNTP, which reformulates point cloud pre-training as a fully causal, decoder-free latent Next-Token Prediction problem.<n>The proposed PointNTP is highly competitive across multiple downstream tasks.
  arXiv  Detail & Related papers  (2026-05-17T17:54:55Z)
• Action-Geometry Prediction with 3D Geometric Prior for Bimanual Manipulation [53.09168514034483]
  Bimanual manipulation requires policies that can reason about 3D geometry, anticipate how it evolves under action, and generate smooth, coordinated motions.<n>We propose a framework that builds bimanual manipulation directly on a pre-trained 3D geometric foundation model.<n>Our policy fuses geometry-aware latents, 2D semantic features, and proprioception into a unified state representation, and uses diffusion model to jointly predict a future action chunk and a future 3D latent that decodes into a dense pointmap.
  arXiv  Detail & Related papers  (2026-02-27T08:54:20Z)
• E3D-Bench: A Benchmark for End-to-End 3D Geometric Foundation Models [78.1674905950243]
  We present the first comprehensive benchmark for 3D geometric foundation models (GFMs)<n>GFMs directly predict dense 3D representations in a single feed-forward pass, eliminating the need for slow or unavailable precomputed camera parameters.<n>We evaluate 16 state-of-the-art GFMs, revealing their strengths and limitations across tasks and domains.<n>All code, evaluation scripts, and processed data will be publicly released to accelerate research in 3D spatial intelligence.
  arXiv  Detail & Related papers  (2025-06-02T17:53:09Z)
• ET-Former: Efficient Triplane Deformable Attention for 3D Semantic Scene Completion From Monocular Camera [53.20087549782785]
  We introduce ET-Former, a novel end-to-end algorithm for semantic scene completion using a single monocular camera.<n>Our approach generates a semantic occupancy map from single RGB observation while simultaneously providing uncertainty estimates for semantic predictions.
  arXiv  Detail & Related papers  (2024-10-14T19:14:49Z)
• SD-Net: Symmetric-Aware Keypoint Prediction and Domain Adaptation for 6D Pose Estimation In Bin-picking Scenarios [2.786599193929693]
  We propose a new 6D pose estimation network with symmetric-aware keypoint prediction and self-training domain adaptation (SD-Net) At the keypoint prediction stage, we designe a robust 3D keypoints selection strategy to locate 3D keypoints even in highly occluded scenes. At the domain adaptation stage, we propose the self-training framework using a student-teacher training scheme. On public Sil'eane dataset, SD-Net achieves state-of-the-art results, obtaining an average precision of 96%.
  arXiv  Detail & Related papers  (2024-03-14T12:08:44Z)
• V-DETR: DETR with Vertex Relative Position Encoding for 3D Object Detection [73.37781484123536]
  We introduce a highly performant 3D object detector for point clouds using the DETR framework. To address the limitation, we introduce a novel 3D Relative Position (3DV-RPE) method. We show exceptional results on the challenging ScanNetV2 benchmark.
  arXiv  Detail & Related papers  (2023-08-08T17:14:14Z)
• CAGroup3D: Class-Aware Grouping for 3D Object Detection on Point Clouds [55.44204039410225]
  We present a novel two-stage fully sparse convolutional 3D object detection framework, named CAGroup3D. Our proposed method first generates some high-quality 3D proposals by leveraging the class-aware local group strategy on the object surface voxels. To recover the features of missed voxels due to incorrect voxel-wise segmentation, we build a fully sparse convolutional RoI pooling module.
  arXiv  Detail & Related papers  (2022-10-09T13:38:48Z)
• BoxGraph: Semantic Place Recognition and Pose Estimation from 3D LiDAR [22.553026961366005]
  We model 3D point clouds as fully-connected graphs of semantically identified components. Optimal association across graphs allows for full 6-Degree-of-Freedom (DoF) pose estimation and place recognition. This representation is very concise, condensing the size of maps by a factor of 25 against the state-of-the-art.
  arXiv  Detail & Related papers  (2022-06-30T09:39:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.