Inside VLM Chart Reading: Tracing Value Reading from Vertical Bar Charts Across Space and Depth

• URL: http://arxiv.org/abs/2609.13745v1
• Date: Sat, 12 Sep 2026 06:28:11 GMT
• Title: Inside VLM Chart Reading: Tracing Value Reading from Vertical Bar Charts Across Space and Depth
• Abstract summary: Vision--language models (VLMs) can answer chart questions accurately, but output accuracy does not show how they combine the evidence needed to recover an exact value.<n>We study vertical-bar value reading with controlled counterfactual activation patching in Qwen2.5VL-7B-Instruct and InternVL3.5-8B.
• Score: 50.62320148925983
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Vision--language models (VLMs) can answer chart questions accurately, but output accuracy does not show how they combine the evidence needed to recover an exact value. We study vertical-bar value reading with controlled counterfactual activation patching in Qwen2.5VL-7B-Instruct and InternVL3.5-8B. The study connects three analyses: (1) The single-factor results show that the changed bar-top region restores much more answer preference than the unchanged bar body, despite containing fewer visual tokens. Legend- and series-related states also lose local recoverability earlier than bar-geometry and axis-scale states. (2) In the handoff analysis, restoration shifts from visual legend regions in early layers to prompt-series positions in middle layers. Resetting the prompt-series state selectively reduces legend-source rescue, supporting its role as a partial mediator. (3) In the factorial analysis, both models can use geometry and scale states from separate donors to favor the combined target. InternVL performs similarly when the states come from separate donors or one image, while Qwen shows lower restoration for separate donors which suggests higher context sensitivity. Together, these results provide preliminary causal evidence for localizing the internal computations that support exact bar-value reading.

Related papers

• Concentrate After Imagination: Text-Conditioned Evidence Grounding for Partially Relevant Video Retrieval [27.64333612353748]
  Partially Relevant Video Retrieval retrieves untrimmed videos when queries describe only short moments.<n>We propose TRACE, a score-level evidence verification operator for PRVR.<n>On ActivityNet Captions, Charades-STA, and TVR, TRACE achieves the best SumR on all three benchmarks.
  arXiv  Detail & Related papers  (2026-09-08T16:37:35Z)
• Agentic Reinforcement Learning with Observation-Calibrated Self-Distillation [18.20369971970953]
  On-Policy Self-Distillation (OPSD) re-scores generated tokens under a privileged replay view to obtain dense, token-level supervision.<n>The resulting support may reflect both the privileged information contained in the replay view and score shifts induced by the replay scaffold.<n>We propose Observation-Calibrated Self-Distillation (OCSD) to resolve this confounding.
  arXiv  Detail & Related papers  (2026-08-05T12:52:14Z)
• Steering Vision-Language Models with Joint Sparse Autoencoders [2.367986397868328]
  We introduce the Joint Sparse Autoencoder (JSAE) to factorize sequence-pooled vision and language activations into shared, interpretable image/caption-level features.<n>Applying JSAE to vision-language models (VLMs) yields representations that are difficult to use as controllable cross-modal steering directions.
  arXiv  Detail & Related papers  (2026-06-24T10:11:04Z)
• SegVG: Transferring Object Bounding Box to Segmentation for Visual Grounding [56.079013202051094]
  We present SegVG, a novel method transfers the box-level annotation as signals to provide an additional pixel-level supervision for Visual Grounding. This approach allows us to iteratively exploit the annotation as signals for both box-level regression and pixel-level segmentation.
  arXiv  Detail & Related papers  (2024-07-03T15:30:45Z)
• Breaking the Frame: Visual Place Recognition by Overlap Prediction [53.17564423756082]
  We propose a novel visual place recognition approach based on overlap prediction, called VOP.<n>VOP proceeds co-visible image sections by obtaining patch-level embeddings using a Vision Transformer backbone.<n>Our approach uses a voting mechanism to assess overlap scores for potential database images.
  arXiv  Detail & Related papers  (2024-06-23T20:00:20Z)
• Noisy-Correspondence Learning for Text-to-Image Person Re-identification [50.07634676709067]
  We propose a novel Robust Dual Embedding method (RDE) to learn robust visual-semantic associations even with noisy correspondences. Our method achieves state-of-the-art results both with and without synthetic noisy correspondences on three datasets.
  arXiv  Detail & Related papers  (2023-08-19T05:34:13Z)
• A Unified BEV Model for Joint Learning of 3D Local Features and Overlap Estimation [12.499361832561634]
  We present a unified bird's-eye view (BEV) model for jointly learning of 3D local features and overlap estimation. Our method significantly outperforms existing methods on overlap prediction, especially in scenes with small overlaps.
  arXiv  Detail & Related papers  (2023-02-28T12:01:16Z)
• One-Shot Object Localization in Medical Images based on Relative Position Regression [17.251097303541002]
  We present a one-shot framework for organ and landmark localization in volumetric medical images. Our main idea comes from that tissues and organs from different human bodies have a similar relative position and context. Experiments on multi-organ localization from head-and-neck (HaN) CT volumes showed that our method acquired competitive performance in real time.
  arXiv  Detail & Related papers  (2020-12-13T11:54:19Z)
• Graph-PCNN: Two Stage Human Pose Estimation with Graph Pose Refinement [54.29252286561449]
  We propose a two-stage graph-based and model-agnostic framework, called Graph-PCNN. In the first stage, heatmap regression network is applied to obtain a rough localization result, and a set of proposal keypoints, called guided points, are sampled. In the second stage, for each guided point, different visual feature is extracted by the localization. The relationship between guided points is explored by the graph pose refinement module to get more accurate localization results.
  arXiv  Detail & Related papers  (2020-07-21T04:59:15Z)
• High-Order Information Matters: Learning Relation and Topology for Occluded Person Re-Identification [84.43394420267794]
  We propose a novel framework by learning high-order relation and topology information for discriminative features and robust alignment. Our framework significantly outperforms state-of-the-art by6.5%mAP scores on Occluded-Duke dataset.
  arXiv  Detail & Related papers  (2020-03-18T12:18:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.