VEIL: How Visual Encoding Hijacking Induces Bias In Vision Models

• URL: http://arxiv.org/abs/2607.05641v1
• Date: Mon, 06 Jul 2026 21:10:08 GMT
• Title: VEIL: How Visual Encoding Hijacking Induces Bias In Vision Models
• Abstract summary: We present VEIL: a systematic study examining how chart encodings influence learned representations.<n>We show that visualization design choices shape learned representations in ways that warrant treating chart-based TSC as a representation and measurement problem.
• Score: 10.461445926991779
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Rendering time series as chart images for CNN-based classification has become increasingly common in time-series classification (TSC). However, it remains unclear whether models learn underlying temporal patterns or rely on encoding-specific visual cues introduced by chart design. We present VEIL: a systematic study examining how chart encodings influence learned representations through complementary analyses of similarity, transferability, and attribution. Attention-guided training appears to mitigate this effect when encoding sensitivity is consistently identified across diagnostics, but provides limited or negative benefit when such signals are absent. These findings position VEIL within the broader question of how machines perceive visualizations -- extending graphical perception from human readers to vision models -- and show that visualization design choices shape learned representations in ways that warrant treating chart-based TSC as a representation and measurement problem rather than a simple modeling decision.

Related papers

• Attention-Guided Saliency Maps for Interpreting Visualization Literacy in VLMs [4.507919939159203]
  We introduce a diagnostic saliency map method tailored for text generation over images using transformer models.<n>Our approach aggregates the language model's attention over the visual tokens across all heads and layers, then maps this attention back onto the vision encoder's patch grid to localise it over the image.
  arXiv  Detail & Related papers  (2026-07-17T16:40:23Z)
• Unveiling the Visual Counting Bottleneck in Vision-Language Models [49.591496870141846]
  This work deconstructs visual counting into three cognitive stages: visual individuation, magnitude awareness, and symbolic mapping.<n>Using synthetic Go boards and linear probes, we demonstrate that visual backbones maintain robust, linearly separable representations of quantity well into the extrapolation regime.<n>We pinpoint the collapse to the symbolic mapping stage, where the model fails to project valid visual magnitudes onto symbolic tokens.
  arXiv  Detail & Related papers  (2026-05-28T16:20:29Z)
• SigTime: Learning and Visually Explaining Time Series Signatures [22.200677868580204]
  We introduce a novel learning framework that jointly trains two Transformer models using complementary time series representations.<n>The learned shapelets serve as interpretable signatures that differentiate time series across classification labels.<n>We develop a visual analytics system -- SigTIme -- with coordinated views to facilitate exploration of time series signatures.
  arXiv  Detail & Related papers  (2025-12-12T22:47:34Z)
• MIRNet: Integrating Constrained Graph-Based Reasoning with Pre-training for Diagnostic Medical Imaging [67.74482877175797]
  MIRNet is a novel framework that integrates self-supervised pre-training with constrained graph-based reasoning.<n>We introduce TongueAtlas-4K, a benchmark comprising 4,000 images annotated with 22 diagnostic labels.
  arXiv  Detail & Related papers  (2025-11-13T06:30:41Z)
• Gaze-Guided Learning: Avoiding Shortcut Bias in Visual Classification [3.1208151315473622]
  We introduce Gaze-CIFAR-10, a human gaze time-series dataset, along with a dual-sequence gaze encoder.<n>In parallel, a Vision Transformer (ViT) is employed to learn the sequential representation of image content.<n>Our framework integrates human gaze priors with machine-derived visual sequences, effectively correcting inaccurate localization in image feature representations.
  arXiv  Detail & Related papers  (2025-04-08T00:40:46Z)
• A Comparative Study of Scanpath Models in Graph-Based Visualization [7.592272924252313]
  Eye-tracking (ET) data presents challenges related to cost, privacy, and scalability.<n>In our study, we conducted an ET experiment with 40 participants who analyzed graphs.<n>We compared human scanpaths with synthetic ones generated by models such as DeepGaze, UMSS, and Gazeformer.
  arXiv  Detail & Related papers  (2025-03-31T14:43:42Z)
• A Survey of Deep Graph Learning under Distribution Shifts: from Graph Out-of-Distribution Generalization to Adaptation [59.14165404728197]
  We provide an up-to-date and forward-looking review of deep graph learning under distribution shifts.<n>Specifically, we cover three primary scenarios: graph OOD generalization, training-time graph OOD adaptation, and test-time graph OOD adaptation.<n>To provide a better understanding of the literature, we introduce a systematic taxonomy that classifies existing methods into model-centric and data-centric approaches.
  arXiv  Detail & Related papers  (2024-10-25T02:39:56Z)
• Assessing Graphical Perception of Image Embedding Models using Channel Effectiveness [20.269583912221734]
  We introduce a novel evaluation framework to assess the graphical perception of image embedding models. For chart comprehension, we examine two main aspects of channel effectiveness: accuracy and discriminability of various visual channels. Experiments with the CLIP model show that it perceives channel accuracy differently from humans and shows unique discriminability in channels like length, tilt, and curvature.
  arXiv  Detail & Related papers  (2024-07-30T14:22:13Z)
• Scaling Laws of Synthetic Images for Model Training ... for Now [54.43596959598466]
  We study the scaling laws of synthetic images generated by state of the art text-to-image models. We observe that synthetic images demonstrate a scaling trend similar to, but slightly less effective than, real images in CLIP training.
  arXiv  Detail & Related papers  (2023-12-07T18:59:59Z)
• Visual Perturbation-aware Collaborative Learning for Overcoming the Language Prior Problem [60.0878532426877]
  We propose a novel collaborative learning scheme from the viewpoint of visual perturbation calibration. Specifically, we devise a visual controller to construct two sorts of curated images with different perturbation extents. The experimental results on two diagnostic VQA-CP benchmark datasets evidently demonstrate its effectiveness.
  arXiv  Detail & Related papers  (2022-07-24T23:50:52Z)
• Deep Co-Attention Network for Multi-View Subspace Learning [73.3450258002607]
  We propose a deep co-attention network for multi-view subspace learning. It aims to extract both the common information and the complementary information in an adversarial setting. In particular, it uses a novel cross reconstruction loss and leverages the label information to guide the construction of the latent representation.
  arXiv  Detail & Related papers  (2021-02-15T18:46:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.