CREG: Compass Relational Evidence for Interpreting Spatial Reasoning in Vision-Language Models

• URL: http://arxiv.org/abs/2603.20475v1
• Date: Fri, 20 Mar 2026 20:09:19 GMT
• Title: CREG: Compass Relational Evidence for Interpreting Spatial Reasoning in Vision-Language Models
• Abstract summary: Vision-language models (VLMs) perform strongly on spatial reasoning benchmarks, yet how they encode directional relations remains poorly understood.<n>We introduce CREG (Grad-Act Graph), a framework that projects multi-layer contrastive spatial attributions into a reference-centered polar system.<n>Our results show that contrastive, multi-layer attribution can expose directional evidence more faithfully than standard saliency-based explanations.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Vision-language models (VLMs) perform strongly on spatial reasoning benchmarks, yet how they encode directional relations remains poorly understood. Existing attribution methods such as GradCAM and attention rollout reveal where a model attends, but not what direction it infers between objects. We introduce CREG (Compass Relational Evidence Graph), a training-free interpretability framework that projects multi-layer contrastive Grad-times-Act attributions into a reference-centered polar coordinate system, producing a directional evidence distribution over compass sectors. To evaluate directional explanations, we propose three metrics: Direction Alignment Error (DAE), Edge Accuracy (EA), and Causal Occlusion Score (COS). On Qwen2-VL-7B across VSR and COCO-Pairs, CREG consistently outperforms standard attribution baselines; on COCO-Pairs, prediction-targeted CREG achieves a DAE of 55.5 degrees and an EA of 0.553, improving over attention rollout by 16.1 degrees in angular error and 0.120 in EA. Causal occlusion experiments on 540 samples across both datasets further support the faithfulness of these directional explanations, with COS greater than or equal to +0.42. The gains are smaller on Qwen2-VL-2B, suggesting that CREG benefits from the more structured spatial representations that emerge at larger scales. Overall, our results show that contrastive, multi-layer attribution can expose directional evidence more faithfully than standard saliency-based explanations in VLM spatial reasoning.

Related papers

• Architecture-Dependent Causal Transfer of Activation States Across Large Language Models [51.56484100374058]
  Internal language activation states can be transferred causally between different large language models.<n>Injecting projected activations into a target model during generation produces a statistically significant, preregistered causal effect.<n>We conclude that end-to-end activation-state transfer between LLMs, as currently implemented, is architecture-dependent rather than universal.
  arXiv  Detail & Related papers  (2026-08-17T09:53:27Z)
• FoundationGeo: Learning Spatial Pixel-Wise Fields for Monocular Metric Geometry [52.232826445242644]
  FoundationGeo bridges relative and metric prediction via spatial calibration and principled data design.<n>Stage 1 learns a high-fidelity, affine-invariant geometry model by initializing with DINOv3.<n>Stage 2 moves beyond global scaling by introducing pixel-wise calibration fields for metric estimation.
  arXiv  Detail & Related papers  (2026-07-13T14:10:01Z)
• EAGOR: Embodied Reasoning in Omni-direction [3.042262674701172]
  We propose EAGOR, a training-free, geometry-aware framework for embodied 360 directional reasoning.<n>EAGOR consistently outperforms existing methods, achieving average relative gains of +34.4% and +45.6% on benchmark datasets.
  arXiv  Detail & Related papers  (2026-07-07T11:39:57Z)
• Why Far Looks Up: Probing Spatial Representation in Vision-Language Models [44.01280555484389]
  Vision-language models (VLMs) achieve strong performance on spatial reasoning benchmarks.<n>But it remains unclear whether this reflects structured 3D understanding or reliance on statistical shortcuts in natural images.<n>We introduce a representation-level analysis framework that constructs minimal contrastive pairs to measure how spatial axes are organized.
  arXiv  Detail & Related papers  (2026-05-28T16:18:01Z)
• Cross-Family Universality of Behavioral Axes via Anchor-Projected Representations [8.041250483194434]
  Large language models from different families use different hidden dimensions, tokenizers, and training procedures.<n>We introduce an anchor-projection framework that maps hidden representations from each model into a shared anchor coordinate space.<n>We find that same-axis directions align tightly across the Llama-Qwen-Mistral-Phi (LQMP) cluster in ACS.
  arXiv  Detail & Related papers  (2026-05-11T02:01:18Z)
• Hyperbolic Enhanced Representation Learning for Incomplete Multi-view Clustering [57.38215918201251]
  We propose HERL, a Hyperbolic Enhanced Representation Learning framework for Incomplete Multi-View Clustering.<n> operating within the Poincaré ball, HERL constructs a structure-aware latent space to enhance representation learning.<n>We show that HERL consistently outperforms state-of-the-art approaches.
  arXiv  Detail & Related papers  (2026-04-18T10:50:46Z)
• On the Role of Reasoning Patterns in the Generalization Discrepancy of Long Chain-of-Thought Supervised Fine-Tuning [63.41902113656453]
  Supervised Fine-Tuning (SFT) on long Chain-of-Thought (CoT) trajectories has become a pivotal phase in building large reasoning models.<n>We conduct a comparative study using two sources of verified CoT trajectories generated by two competing models.<n>We find that SFT on textttDeepSeek-R1-0528 data achieves remarkably lower training loss, yet exhibits significantly worse generalization performance.
  arXiv  Detail & Related papers  (2026-04-02T07:00:54Z)
• m2sv: A Scalable Benchmark for Map-to-Street-View Spatial Reasoning [1.2162625261919089]
  We introduce m2sv, a scalable benchmark for map-to-street-view spatial reasoning.<n>We release m2sv-20k, a geographically diverse benchmark with controlled ambiguity, along with m2sv-sft-11k, a curated set of structured reasoning traces for supervised fine-tuning.<n>Despite strong performance on existing multimodal benchmarks, the best evaluated VLM achieves only 65.2% accuracy on m2sv, far below the human baseline of 95%.
  arXiv  Detail & Related papers  (2026-01-27T02:01:56Z)
• SEAL: Steerable Reasoning Calibration of Large Language Models for Free [58.931194824519935]
  Large Language Models (LLMs) have demonstrated compelling capabilities for complex reasoning tasks via the extended chain-of-thought (CoT) reasoning mechanism.<n>Recent studies reveal substantial redundancy in the CoT reasoning traces, which negatively impacts model performance.<n>We introduce SEAL, a training-free approach that seamlessly calibrates the CoT process, improving accuracy while demonstrating significant efficiency gains.
  arXiv  Detail & Related papers  (2025-04-07T02:42:07Z)
• SURDS: Benchmarking Spatial Understanding and Reasoning in Driving Scenarios with Vision Language Models [15.50826328938879]
  We introduce SURDS, a benchmark designed to evaluate the spatial reasoning capabilities of vision language models (VLMs)<n>Built on the nuScenes dataset, SURDS comprises 41,080 vision-question-answer training instances and 9,250 evaluation samples.<n>We propose a reinforcement learning-based alignment scheme leveraging spatially grounded reward signals.
  arXiv  Detail & Related papers  (2024-11-20T08:14:01Z)
• Relation DETR: Exploring Explicit Position Relation Prior for Object Detection [26.03892270020559]
  We present a scheme for enhancing the convergence and performance of DETR (DEtection TRansformer) Our approach, termed Relation-DETR, introduces an encoder to construct position relation embeddings for progressive attention refinement. Experiments on both generic and task-specific datasets demonstrate the effectiveness of our approach.
  arXiv  Detail & Related papers  (2024-07-16T13:17:07Z)
• Revisiting Link Prediction: A Data Perspective [59.296773787387224]
  Link prediction, a fundamental task on graphs, has proven indispensable in various applications, e.g., friend recommendation, protein analysis, and drug interaction prediction. Evidence in existing literature underscores the absence of a universally best algorithm suitable for all datasets. We recognize three fundamental factors critical to link prediction: local structural proximity, global structural proximity, and feature proximity.
  arXiv  Detail & Related papers  (2023-10-01T21:09:59Z)
• Quaternion-valued Correlation Learning for Few-Shot Semantic Segmentation [33.88445464404075]
  Few-shot segmentation (FSS) aims to segment unseen classes given only a few samples. We introduce a quaternion perspective on correlation learning and propose a novel Quaternion-valued Correlation Learning Network (QCLNet) Our QCLNet is formulated as a hyper-complex valued network and represents correlation tensors in the quaternion domain, which uses quaternion-valued convolution to explore the external relations of query subspace.
  arXiv  Detail & Related papers  (2023-05-12T06:56:22Z)
• Robust Self-Supervised LiDAR Odometry via Representative Structure Discovery and 3D Inherent Error Modeling [67.75095378830694]
  We develop a two-stage odometry estimation network, where we obtain the ego-motion by estimating a set of sub-region transformations. In this paper, we aim to alleviate the influence of unreliable structures in training, inference and mapping phases. Our two-frame odometry outperforms the previous state of the arts by 16%/12% in terms of translational/rotational errors.
  arXiv  Detail & Related papers  (2022-02-27T12:52:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.