Visible Yet Unreadable: A Systematic Blind Spot of Vision Language Models Across Writing Systems

• URL: http://arxiv.org/abs/2509.06996v4
• Date: Tue, 21 Oct 2025 04:49:38 GMT
• Title: Visible Yet Unreadable: A Systematic Blind Spot of Vision Language Models Across Writing Systems
• Abstract summary: We investigate whether advanced vision language models (VLMs) share this resilience.<n>We construct two psychophysics inspired benchmarks across distinct writing systems.<n>Despite strong performance on clean text, contemporary VLMs show a severe drop under these perturbations.
• Score: 25.47053654117902
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Writing is a universal cultural technology that reuses vision for symbolic communication. Humans display striking resilience: we readily recognize words even when characters are fragmented, fused, or partially occluded. This paper investigates whether advanced vision language models (VLMs) share this resilience. We construct two psychophysics inspired benchmarks across distinct writing systems, Chinese logographs and English alphabetic words, by splicing, recombining, and overlaying glyphs to yield ''visible but unreadable'' stimuli for models while remaining legible to humans. Despite strong performance on clean text, contemporary VLMs show a severe drop under these perturbations, frequently producing unrelated or incoherent outputs. The pattern suggests a structural limitation: models heavily leverage generic visual invariances but under rely on compositional priors needed for robust literacy. We release stimuli generation code, prompts, and evaluation protocols to facilitate transparent replication and follow up work. Our findings motivate architectures and training strategies that encode symbol segmentation, composition, and binding across scripts, and they delineate concrete challenges for deploying multimodal systems in education, accessibility, cultural heritage, and security.

Related papers

• Cognitive Mismatch in Multimodal Large Language Models for Discrete Symbol Understanding [96.81411333150213]
  This paper introduces a benchmark to evaluate how top-tier MLLMs navigate discrete semantic spaces.<n>Our investigation uncovers a counterintuitive phenomenon: models often fail at basic symbol recognition yet succeed in complex reasoning tasks.<n>This work offers a roadmap for developing more rigorous, human-aligned intelligent systems.
  arXiv  Detail & Related papers  (2026-03-19T04:08:20Z)
• LogoDiffuser: Training-Free Multilingual Logo Generation and Stylization via Letter-Aware Attention Control [6.75235033060142]
  We propose a training-free method that synthesizes multilingual logo designs using the multimodal diffusion transformer.<n>Instead of using textual prompts, we input the target characters as images, enabling robust character structure control regardless of language.<n>Our method achieves state-of-the-art performance in multilingual logo generation.
  arXiv  Detail & Related papers  (2026-03-10T14:57:46Z)
• Reading $\ eq$ Seeing: Diagnosing and Closing the Typography Gap in Vision-Language Models [16.13083848552738]
  Vision-Language Models achieve near-perfect accuracy at reading text in images, yet prove largely typography-blind.<n>We systematically investigate this gap by evaluating font family, size, style, and color recognition across 26 fonts, four scripts, and three difficulty levels.<n>We release our evaluation framework, data, and fine-tuning recipe to support progress in closing the typographic gap in vision-language understanding.
  arXiv  Detail & Related papers  (2026-03-09T15:31:47Z)
• Eye-Q: A Multilingual Benchmark for Visual Word Puzzle Solving and Image-to-Phrase Reasoning [1.6234264741872295]
  Vision-Language Models (VLMs) have achieved strong performance on standard vision-language benchmarks.<n>We propose visual word puzzles as a challenging alternative, as they require discovering implicit visual cues, generating and revising hypotheses, and mapping evidence to non-literal concepts.<n>We introduce Eye-Q, a multilingual benchmark designed to assess this form of complex visual understanding.
  arXiv  Detail & Related papers  (2026-01-06T20:27:29Z)
• See the Text: From Tokenization to Visual Reading [63.10220471118435]
  SeeTok renders text as images (visual-text) and leverages pretrained multimodal computations to interpret them.<n>Across three different language tasks, SeeTok matches or surpasses subword tokenizers while requiring 4.43 times fewer tokens and reducing FLOPs by 70.5%.<n>SeeTok signals a shift from symbolic tokenization to human-like visual reading, and takes a step toward more natural and cognitively inspired language models.
  arXiv  Detail & Related papers  (2025-10-21T17:34:48Z)
• ViCrit: A Verifiable Reinforcement Learning Proxy Task for Visual Perception in VLMs [98.27348724529257]
  We introduce ViCrit (Visual Caption Hallucination Critic), an RL proxy task that trains VLMs to localize a subtle, synthetic visual hallucination injected into paragraphs of human-written image captions.<n>Models trained with the ViCrit Task exhibit substantial gains across a variety of vision-language models benchmarks.
  arXiv  Detail & Related papers  (2025-06-11T19:16:54Z)
• Text Speaks Louder than Vision: ASCII Art Reveals Textual Biases in Vision-Language Models [93.46875303598577]
  Vision-language models (VLMs) have advanced rapidly in processing multimodal information, but their ability to reconcile conflicting signals remains underexplored.<n>This work investigates how VLMs process ASCII art, a unique medium where textual elements collectively form visual patterns, potentially creating semantic-visual conflicts.
  arXiv  Detail & Related papers  (2025-04-02T10:47:07Z)
• Empowering Backbone Models for Visual Text Generation with Input Granularity Control and Glyph-Aware Training [68.41837295318152]
  Diffusion-based text-to-image models have demonstrated impressive achievements in diversity and aesthetics but struggle to generate images with visual texts. Existing backbone models have limitations such as misspelling, failing to generate texts, and lack of support for Chinese text. We propose a series of methods, aiming to empower backbone models to generate visual texts in English and Chinese.
  arXiv  Detail & Related papers  (2024-10-06T10:25:39Z)
• Unified Language-Vision Pretraining in LLM with Dynamic Discrete Visual Tokenization [52.935150075484074]
  We introduce a well-designed visual tokenizer to translate the non-linguistic image into a sequence of discrete tokens like a foreign language. The resulting visual tokens encompass high-level semantics worthy of a word and also support dynamic sequence length varying from the image. This unification empowers LaVIT to serve as an impressive generalist interface to understand and generate multi-modal content simultaneously.
  arXiv  Detail & Related papers  (2023-09-09T03:01:38Z)
• Visually-Situated Natural Language Understanding with Contrastive Reading Model and Frozen Large Language Models [24.456117679941816]
  Contrastive Reading Model (Cream) is a novel neural architecture designed to enhance the language-image understanding capability of Large Language Models (LLMs) Our approach bridges the gap between vision and language understanding, paving the way for the development of more sophisticated Document Intelligence Assistants.
  arXiv  Detail & Related papers  (2023-05-24T11:59:13Z)
• Probing Contextual Language Models for Common Ground with Visual Representations [76.05769268286038]
  We design a probing model that evaluates how effective are text-only representations in distinguishing between matching and non-matching visual representations. Our findings show that language representations alone provide a strong signal for retrieving image patches from the correct object categories. Visually grounded language models slightly outperform text-only language models in instance retrieval, but greatly under-perform humans.
  arXiv  Detail & Related papers  (2020-05-01T21:28:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.