More Than Where You Are: Learning Semantics, Structure, and Geometry from Cross-View Localization

• URL: http://arxiv.org/abs/2607.12429v1
• Date: Tue, 14 Jul 2026 07:00:34 GMT
• Title: More Than Where You Are: Learning Semantics, Structure, and Geometry from Cross-View Localization
• Abstract summary: Cross-view understanding under extreme viewpoint changes is essential for spatial intelligence.<n>We argue that cross-view localization should not be viewed merely as 2D matching or pose estimation.<n>We propose CROSS, a unified cross-view localization framework built upon 3D-grounded alignment, structure-aware matching, and hypothesis ranking.
• Score: 8.17466031886622
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Consistent cross-view understanding under extreme viewpoint changes is essential for spatial intelligence, as it enables models to recognize the same scene across extreme viewpoint gaps. Cross-view localization naturally provides a promising pathway toward this ability, as it requires a model to align ground-view imagery with geo-referenced satellite-view imagery despite drastic appearance changes to estimate camera poses. Recent visual foundation models have made this long-standing localization problem increasingly feasible by providing rich 2D representations for cross-view matching. However, we argue that cross-view localization should not be viewed merely as 2D matching or pose estimation. In this work, we revisit cross-view localization as more than pose estimation and investigate how it can help the model develop consistent cross-view understanding under extreme viewpoint changes, including stable semantics, reliable structure, and transferable geometry. We identify three key limitations of existing methods that prevent them from achieving this. They usually lack explicit 3D grounding, rely on strict point-wise matching that can weaken semantic consistency, and learn from an absolute objective that provides limited guidance for geometric reasoning. To address these limitations, we propose CROSS, a unified cross-view localization framework built upon 3D-grounded alignment, structure-aware matching, and hypothesis ranking. This formulation makes structure learning an intrinsic requirement, encourages semantic representations to remain stable, and enables the model to acquire transferable geometry. Extensive experiments on the KITTI and VIGOR datasets show that CROSS achieves state-of-the-art performance in cross-view localization. More importantly, CROSS effectively learns stable semantics, reliable structure, and transferable geometry across extremely different viewpoints.

Related papers

• DDMS: Discriminative Distillation of Multi-view Foundational Features into Single-view Models [17.93776710445663]
  We demonstrate that by re-distilling multi-view models into a single-view estimator, we can obtain enhanced 3D consistent foundational features.<n>Our key idea is to construct a multi-view teacher by fusing pretrained 2D foundation features with multi-view geometric features.<n>Our method consistently produces stronger 3D-aware foundation features that improve multi-view consistency and local discriminability.
  arXiv  Detail & Related papers  (2026-08-24T21:38:49Z)
• ProjFormer: Point Cloud Completion via Geometric-Projective Transformer and Cross-Modal Semantic Constraints [50.4059685962603]
  ProjFormer is a cross-modal framework that enforces geometry-consistent 2D-3D interaction through explicit projection and adaptive feature routing.<n> Experiments show that, under a lightweight design, ProjFormer delivers competitive performance with improved structural completeness.
  arXiv  Detail & Related papers  (2026-08-15T08:05:48Z)
• G$^2$TAM: Geometry Grounded Track Anything Model [49.05553245076823]
  We propose a unified framework for promptable instance tracking in 3D using only unordered RGB images or videos.<n>G$2$TAM employs spatially aligned geometric representations as implicit memory, ensuring stable instance identity and localization across frames and views.<n>At its core is a cross-modal spatial encoder that integrates visual and textual prompts into a shared geometric space, enabling end-to-end spatial reconstruction and instance-consistent mask prediction.
  arXiv  Detail & Related papers  (2026-07-04T09:32:13Z)
• Beyond 2D Matching: A Unified Single-Stage Framework for Geometry-Aware Cross-View Object Geo-Localization [21.013795754583697]
  Cross-view object geo-localization (CVOGL) aims to locate a target object from a query view within a geo-tagged reference image (e.g., satellite)<n>Existing approaches heavily rely on 2D appearance matching and are constrained by limited datasets lacking geometric metadata.<n>We first introduce dataset, a large-scale, high-fidelity building dataset comprising over 220,000 ground-satellite and drone-satellite pairs.<n>We propose a novel single-stage Geometry-Aware Geo-localization framework (GAGeo), built upon the permutation-equivariant 3D foundation model $
  arXiv  Detail & Related papers  (2026-06-29T17:19:49Z)
• VistaRef: Boosting Visual Spatial Orientation Awareness for Pointing-to-Object Detection [19.163908796279802]
  Grounding deictic gestures in natural images is fundamental to AR and human-robot collaboration.<n> VistaRef is a framework designed to explicitly enhance spatial orientation awareness.
  arXiv  Detail & Related papers  (2026-06-23T12:30:04Z)
• SpatialEvo: Self-Evolving Spatial Intelligence via Deterministic Geometric Environments [75.60795462502949]
  SpatialEvo is a self-evolving framework for 3D spatial reasoning.<n>It formalizes 16 spatial reasoning task categories under explicit geometric validation rules.<n>It converts unannotated 3D scenes into zero-noise interactive oracles, replacing model consensus with objective physical feedback.
  arXiv  Detail & Related papers  (2026-04-15T17:59:12Z)
• Towards Foundation Models for 3D Scene Understanding: Instance-Aware Self-Supervised Learning for Point Clouds [53.82500407523346]
  PointINS is an instance-oriented self-supervised framework that enriches point cloud representations through geometry-aware learning.<n>PointINS achieves on average +3.5% mAP improvement for indoor instance segmentation and +4.1% PQ gain for outdoor panoptic segmentation.
  arXiv  Detail & Related papers  (2026-03-26T08:31:06Z)
• Object Pose Transformer: Unifying Unseen Object Pose Estimation [54.20344997573707]
  Learning model-free object pose estimation for unseen instances remains a fundamental challenge in 3D vision.<n>ours jointly predicts depth, point maps, camera parameters, and normalized object coordinates from RGB inputs.<n>ours is camera-agnostic, learning camera intrinsics on-the-fly and supporting optional depth input for metric-scale recovery.
  arXiv  Detail & Related papers  (2026-03-24T16:04:16Z)
• ControlVP: Interactive Geometric Refinement of AI-Generated Images with Consistent Vanishing Points [32.23473666846317]
  We propose ControlVP, a user-guided framework for correcting vanishing point inconsistencies in generated images.<n>Our approach extends a pre-trained diffusion model by incorporating structural guidance derived from building contours.<n>Our method enhances global geometric consistency while maintaining visual fidelity comparable to the baselines.
  arXiv  Detail & Related papers  (2025-12-08T12:38:11Z)
• TALO: Pushing 3D Vision Foundation Models Towards Globally Consistent Online Reconstruction [57.46712611558817]
  3D vision foundation models have shown strong generalization in reconstructing key 3D attributes from uncalibrated images through a single feed-forward pass.<n>Recent strategies align consecutive predictions by solving global transformation, yet our analysis reveals their fundamental limitations in assumption validity, local alignment scope, and robustness under noisy geometry.<n>We propose a higher-DOF and long-term alignment framework based on Thin Plate Spline, leveraging globally propagated control points to correct spatially varying inconsistencies.
  arXiv  Detail & Related papers  (2025-12-02T02:22:20Z)
• Revisiting Cross-View Localization from Image Matching [12.411420734642988]
  Cross-view localization aims to estimate the 3 degrees of freedom pose of a ground-view image by registering it to aerial or satellite imagery.<n>Existing methods either regress poses directly or align features in a shared bird's-eye view (BEV) space.<n>We propose a novel framework that improves both matching and localization.
  arXiv  Detail & Related papers  (2025-08-14T14:57:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.