DINOcular: Self-Supervised Visuospatial Representations

• URL: http://arxiv.org/abs/2608.27226v2
• Date: Thu, 03 Sep 2026 11:22:08 GMT
• Title: DINOcular: Self-Supervised Visuospatial Representations
• Abstract summary: We introduce a self-supervised framework for learning joint visuospatial representations from RGB-D observations.<n>Our method integrates depth-derived geometric priors with a visual backbone through inter-patch and intra-patch fusion.<n>The resulting representation shows promising improvements on 3D awareness while preserving semantic transfer.
• Score: 7.360872501623849
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We introduce a self-supervised framework for learning joint visuospatial representations from RGB-D observations. While modern vision foundation models are trained almost exclusively on RGB images, many embodied systems have access to explicit depth sensing, which provides geometric information that monocular inputs cannot recover. Our method integrates depth-derived geometric priors with a visual backbone through inter-patch and intra-patch fusion, enabling the model to encode both appearance and spatial structure efficiently. The resulting representation shows promising improvements on 3D awareness while preserving semantic transfer: it outperforms prior methods of comparable scale on multiple 3D geometry benchmarks, and remains competitive when probed for standard RGB-D semantic segmentation tasks.

Related papers

• GA-VLN: Geometry-Aware BEV Representation for Efficient Vision-Language Navigation [75.85672467847631]
  We introduce the Geometry-Aware BEV (GA-BEV) - a compact, 3D-grounded feature representation.<n>We construct BEV spatial maps from RGB-D inputs by projecting visual features into 3D space.<n>We incorporate features from a pretrained 3D foundation model into the BEV space, injecting structural priors learned from large-scale 3D reconstruction tasks.
  arXiv  Detail & Related papers  (2026-05-21T06:20:17Z)
• Align3D-AD: Cross-Modal Feature Alignment and Dual-Prompt Learning for Zero-shot 3D Anomaly Detection [2.08058961865456]
  Zero-shot 3D anomaly detection aims to identify anomalies without access to training data from target categories.<n>Existing methods mainly rely on projecting 3D observations into multi-view representations that primarily capture geometric cues.<n>We propose Align3D-AD, a unified two-stage framework that leverages the RGB modality from auxiliary categories as cross-modal guidance.
  arXiv  Detail & Related papers  (2026-05-07T08:24:44Z)
• Learning 3D Representations for Spatial Intelligence from Unposed Multi-View Images [81.94999489820974]
  UniSplat is a feed-forward framework for learning 3D representations from unposed multi-view images.<n>We introduce a dual-masking strategy that strengthens geometry induction in the encoder.<n>Second, we develop a coarse-to-fine Gaussian splatting strategy that reduces appearance-semantics inconsistencies.<n>Third, we introduce a pose-conditioned recalibration mechanism that interrelates predicted 3D point and semantic maps into the image plane.
  arXiv  Detail & Related papers  (2026-04-12T10:36:18Z)
• Demo-Pose: Depth-Monocular Modality Fusion For Object Pose Estimation [5.466547563815996]
  We address the challenge of category-level 9-DoF pose estimation from RGB-D input, without relying on CAD models during inference.<n>We propose DeMo-Pose, a hybrid architecture that fuses semantic features with depth-based graph convolutional representations.<n>Our approach achieves real-time inference and significantly improves over state-of-the-art methods across object categories.
  arXiv  Detail & Related papers  (2026-03-29T05:58:04Z)
• Learning Human Visual Attention on 3D Surfaces through Geometry-Queried Semantic Priors [0.0]
  We introduce SemGeo-AttentionNet, a dual-stream architecture that formalizes the interplay between geometric processing and semantic recognition.<n>We extend our framework to temporal scanpath generation through reinforcement learning.<n> Evaluation on SAL3D, NUS3D and 3DVA datasets demonstrates substantial improvements.
  arXiv  Detail & Related papers  (2026-02-06T06:15:38Z)
• Joint Geometry-Appearance Human Reconstruction in a Unified Latent Space via Bridge Diffusion [57.09673862519791]
  This paper introduces textbfJGA-LBD, a novel framework that unifies the modeling of geometry and appearance into a joint latent representation.<n> Experiments demonstrate that JGA-LBD outperforms current state-of-the-art approaches in terms of both geometry fidelity and appearance quality.
  arXiv  Detail & Related papers  (2026-01-01T12:48:56Z)
• GLaD: Geometric Latent Distillation for Vision-Language-Action Models [106.53332923530245]
  We introduce GLaD, a geometry-aware Vision-Language-Action (VLA) framework that incorporates 3D geometric priors during pretraining through knowledge distillation.<n>GLaD achieves 94.1% average success rate across four LIBERO task suites, outperforming UniVLA (92.5%) which uses identical pretraining data.
  arXiv  Detail & Related papers  (2025-12-10T13:07:27Z)
• Towards Human-Level 3D Relative Pose Estimation: Generalizable, Training-Free, with Single Reference [65.42565481489132]
  Humans can easily deduce the relative pose of a previously unseen object, without labeling or training, given only a single query-reference image pair.<n>We propose a novel 3D generalizable relative pose estimation method by elaborating 3D/2.5D shape perception with a 2.5D shape from RGB-D reference.<n>Our differentiable takes the 2.5D rotatable mesh textured by the RGB and the semantic maps (obtained by DINOv2 from the RGB input), then renders new RGB and semantic maps under a novel rotated view.
  arXiv  Detail & Related papers  (2024-06-26T16:01:10Z)
• Confidence-Aware RGB-D Face Recognition via Virtual Depth Synthesis [48.59382455101753]
  2D face recognition encounters challenges in unconstrained environments due to varying illumination, occlusion, and pose. Recent studies focus on RGB-D face recognition to improve robustness by incorporating depth information. In this work, we first construct a diverse depth dataset generated by 3D Morphable Models for depth model pre-training. Then, we propose a domain-independent pre-training framework that utilizes readily available pre-trained RGB and depth models to separately perform face recognition without needing additional paired data for retraining.
  arXiv  Detail & Related papers  (2024-03-11T09:12:24Z)
• Geometric-aware Pretraining for Vision-centric 3D Object Detection [77.7979088689944]
  We propose a novel geometric-aware pretraining framework called GAPretrain. GAPretrain serves as a plug-and-play solution that can be flexibly applied to multiple state-of-the-art detectors. We achieve 46.2 mAP and 55.5 NDS on the nuScenes val set using the BEVFormer method, with a gain of 2.7 and 2.1 points, respectively.
  arXiv  Detail & Related papers  (2023-04-06T14:33:05Z)
• Exploring intermediate representation for monocular vehicle pose estimation [38.85309013717312]
  We present a new learning-based framework to recover vehicle pose in SO(3) from a single RGB image. In contrast to previous works that map from local appearance to observation angles, we explore a progressive approach by extracting meaningful Intermediate Geometrical Representations (IGRs) This approach features a deep model that transforms perceived intensities to IGRs, which are mapped to a 3D representation encoding object orientation in the camera coordinate system.
  arXiv  Detail & Related papers  (2020-11-17T06:30:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.