VistaRef: Boosting Visual Spatial Orientation Awareness for Pointing-to-Object Detection

• URL: http://arxiv.org/abs/2606.24498v1
• Date: Tue, 23 Jun 2026 12:30:04 GMT
• Title: VistaRef: Boosting Visual Spatial Orientation Awareness for Pointing-to-Object Detection
• Abstract summary: Grounding deictic gestures in natural images is fundamental to AR and human-robot collaboration.<n> VistaRef is a framework designed to explicitly enhance spatial orientation awareness.
• Score: 19.163908796279802
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Grounding deictic gestures in natural images is fundamental to AR and human-robot collaboration, providing a basis for seamless spatial interaction. While Transformer-based visual models have achieved significant progress in general object detection, their global attention mechanisms often neglect micro-geometric relationships, degrading orientation accuracy. In pointing tasks, this deficiency manifests as an inability to accurately capture the pointing ray implied by finger poses, which results in pointing drift and localization ambiguity when dealing with distant or densely packed objects. To address this, we propose VistaRef, a framework designed to explicitly enhance spatial orientation awareness. First, we develop the Local Hand Entity Modeling (LHEM) module, which incorporates hand-pose embeddings to strengthen the model's capability to capture subtle finger deviations. Second, drawing inspiration from multi-view geometry, we construct the Geometric Ray Modeling (GRM) module to transform implicit orientation information into explicit spatial geometric features, guiding feature aggregation and deep fusion via attention mechanisms. Furthermore, we introduce a novel Orientation-Consistent Alignment Loss (OCAL) to synergistically supervise hand presence and pointing consistency, ensuring that all architectural improvements collectively serve the core objective of spatial localization. Experimental results demonstrate that VistaRef significantly outperforms the baseline, achieving a 14-point absolute gain in grounding accuracy. Qualitative analysis further confirms that VistaRef effectively models the geometric correlation from hand to target, bridging the spatial perception gap inherent in traditional Transformers for complex scenarios. Code: https://github.com/lingli1724/VistaRef.

Related papers

• DyG$^2$T: Modeling Object Dynamics with 3D Gaussian Temporal-Spatial Particle Graph Transformer [86.4618122245946]
  DyG$2$T is a dynamics modeling framework that infers object motion trajectories by spatially completing and temporally discriminating Key Point representations.<n>Experiments on both synthetic and real-world datasets demonstrate that DyG$2$T achieves accurate dynamics modeling and reasoning.
  arXiv  Detail & Related papers  (2026-08-19T03:44:28Z)
• More Than Where You Are: Learning Semantics, Structure, and Geometry from Cross-View Localization [8.17466031886622]
  Cross-view understanding under extreme viewpoint changes is essential for spatial intelligence.<n>We argue that cross-view localization should not be viewed merely as 2D matching or pose estimation.<n>We propose CROSS, a unified cross-view localization framework built upon 3D-grounded alignment, structure-aware matching, and hypothesis ranking.
  arXiv  Detail & Related papers  (2026-07-14T07:00:34Z)
• PointVG-R: Internalizing Geometric Reasoning in MLLMs for Precise Pointing Localization via Visual Chain of Thought [19.680475189691965]
  Pointing-based visual grounding requires models to precisely locate target objects by deciphering complex spatial relationships between the visual scene and pointing gestures.<n>We propose PointVG-R, a reasoning-guided Multi-modal Large Language Model (MLLM)<n>We design a novel geometric reasoning pipeline that simulates the iterative cognitive process humans employ when interpreting pointing gestures.
  arXiv  Detail & Related papers  (2026-06-23T13:06:51Z)
• Towards Foundation Models for 3D Scene Understanding: Instance-Aware Self-Supervised Learning for Point Clouds [53.82500407523346]
  PointINS is an instance-oriented self-supervised framework that enriches point cloud representations through geometry-aware learning.<n>PointINS achieves on average +3.5% mAP improvement for indoor instance segmentation and +4.1% PQ gain for outdoor panoptic segmentation.
  arXiv  Detail & Related papers  (2026-03-26T08:31:06Z)
• RDD: Robust Feature Detector and Descriptor using Deformable Transformer [8.01082121187363]
  We present Robust Deformable Detector (RDD), a novel and robust keypoint detector/descriptor.<n>We observed that deformable attention focuses on key locations, effectively reducing the search space complexity.<n>Our proposed method outperforms all state-of-the-art keypoint detection/description methods in sparse matching tasks.
  arXiv  Detail & Related papers  (2025-05-12T19:24:45Z)
• Guiding Human-Object Interactions with Rich Geometry and Relations [21.528466852204627]
  Existing methods often rely on simplified object representations, such as the object's centroid or the nearest point to a human, to achieve physically plausible motions.<n>We introduce ROG, a novel framework that addresses relationships inherent in HOIs with rich geometric detail.<n>We show that ROG significantly outperforms state-of-the-art methods in the realism evaluations and semantic accuracy of synthesized HOIs.
  arXiv  Detail & Related papers  (2025-03-26T02:57:18Z)
• Boosting Cross-Domain Point Classification via Distilling Relational Priors from 2D Transformers [59.0181939916084]
  Traditional 3D networks mainly focus on local geometric details and ignore the topological structure between local geometries. We propose a novel Priors Distillation (RPD) method to extract priors from the well-trained transformers on massive images. Experiments on the PointDA-10 and the Sim-to-Real datasets verify that the proposed method consistently achieves the state-of-the-art performance of UDA for point cloud classification.
  arXiv  Detail & Related papers  (2024-07-26T06:29:09Z)
• Geometric Features Enhanced Human-Object Interaction Detection [11.513009304308724]
  We propose a novel end-to-end Transformer-style HOI detection model, i.e., geometric features enhanced HOI detector (GeoHOI) One key part of the model is a new unified self-supervised keypoint learning method named UniPointNet. GeoHOI effectively upgrades a Transformer-based HOI detector benefiting from the keypoints similarities measuring the likelihood of human-object interactions.
  arXiv  Detail & Related papers  (2024-06-26T18:52:53Z)
• PointOBB: Learning Oriented Object Detection via Single Point Supervision [55.88982271340328]
  This paper proposes PointOBB, the first single Point-based OBB generation method, for oriented object detection. PointOBB operates through the collaborative utilization of three distinctive views: an original view, a resized view, and a rotated/flipped (rot/flp) view. Experimental results on the DIOR-R and DOTA-v1.0 datasets demonstrate that PointOBB achieves promising performance.
  arXiv  Detail & Related papers  (2023-11-23T15:51:50Z)
• Weakly-supervised Contrastive Learning for Unsupervised Object Discovery [52.696041556640516]
  Unsupervised object discovery is promising due to its ability to discover objects in a generic manner. We design a semantic-guided self-supervised learning model to extract high-level semantic features from images. We introduce Principal Component Analysis (PCA) to localize object regions.
  arXiv  Detail & Related papers  (2023-07-07T04:03:48Z)
• Centralized Feature Pyramid for Object Detection [53.501796194901964]
  Visual feature pyramid has shown its superiority in both effectiveness and efficiency in a wide range of applications. In this paper, we propose a OLO Feature Pyramid for object detection, which is based on a globally explicit centralized feature regulation.
  arXiv  Detail & Related papers  (2022-10-05T08:32:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.