Symbolic Grounding Reveals Representational Bottlenecks in Abstract Visual Reasoning

• URL: http://arxiv.org/abs/2604.21346v1
• Date: Thu, 23 Apr 2026 07:03:48 GMT
• Title: Symbolic Grounding Reveals Representational Bottlenecks in Abstract Visual Reasoning
• Abstract summary: Vision--language models (VLMs) often fail on abstract visual reasoning benchmarks such as Bongard problems.<n>We reformulate Bongard-LOGO as a symbolic reasoning task based on LOGO-style action programs or structured descriptions.
• Score: 2.6641834518599303
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Vision--language models (VLMs) often fail on abstract visual reasoning benchmarks such as Bongard problems, raising the question of whether the main bottleneck lies in reasoning or representation. We study this on Bongard-LOGO, a synthetic benchmark of abstract concept learning with ground-truth generative programs, by comparing end-to-end VLMs on raw images with large language models (LLMs) given symbolic inputs derived from those images. Using symbolic inputs as a diagnostic probe rather than a practical multimodal architecture, our \emph{Componential--Grammatical (C--G)} paradigm reformulates Bongard-LOGO as a symbolic reasoning task based on LOGO-style action programs or structured descriptions. LLMs achieve large and consistent gains, reaching mid--90s accuracy on Free-form problems, while a strong visual baseline remains near chance under matched task definitions. Ablations on input format, explicit concept prompts, and minimal visual grounding show that these factors matter much less than the shift from pixels to symbolic structure. These results identify representation as a key bottleneck in abstract visual reasoning and show how symbolic input can serve as a controlled diagnostic upper bound.

Related papers

• From Frames to Temporal Graphs: In-Context Egocentric Action Recognition with Vision-Language Models [67.00247936789127]
  Action reasoning in egocentric video requires capturing fine-grained transitions of hand-object interactions.<n>We propose to decouple visual perception from symbolic reasoning by converting videos into Temporal Action Graphs.
  arXiv  Detail & Related papers  (2026-06-13T18:00:42Z)
• Unveiling the Visual Counting Bottleneck in Vision-Language Models [49.591496870141846]
  This work deconstructs visual counting into three cognitive stages: visual individuation, magnitude awareness, and symbolic mapping.<n>Using synthetic Go boards and linear probes, we demonstrate that visual backbones maintain robust, linearly separable representations of quantity well into the extrapolation regime.<n>We pinpoint the collapse to the symbolic mapping stage, where the model fails to project valid visual magnitudes onto symbolic tokens.
  arXiv  Detail & Related papers  (2026-05-28T16:20:29Z)
• ViCrit: A Verifiable Reinforcement Learning Proxy Task for Visual Perception in VLMs [98.27348724529257]
  We introduce ViCrit (Visual Caption Hallucination Critic), an RL proxy task that trains VLMs to localize a subtle, synthetic visual hallucination injected into paragraphs of human-written image captions.<n>Models trained with the ViCrit Task exhibit substantial gains across a variety of vision-language models benchmarks.
  arXiv  Detail & Related papers  (2025-06-11T19:16:54Z)
• VisuRiddles: Fine-grained Perception is a Primary Bottleneck for Multimodal Large Language Models in Abstract Visual Reasoning [70.44416154144001]
  Recent strides in multimodal large language models (MLLMs) have significantly advanced their performance in many reasoning tasks.<n> Abstract Visual Reasoning (AVR) remains a critical challenge, primarily due to limitations in perceiving abstract graphics.<n>We propose VisuRiddles, a benchmark for PRS, featuring tasks meticulously constructed to assess models' reasoning capacities.<n>Second, we introduce the Perceptual Riddle Synthesizer (PRS), an automated framework for generating riddles with fine-grained perceptual descriptions.
  arXiv  Detail & Related papers  (2025-06-03T07:24:00Z)
• VisualSphinx: Large-Scale Synthetic Vision Logic Puzzles for RL [11.10804309162152]
  We propose VisualSphinx, a first-of-its-kind large-scale synthetic visual logical reasoning training data.<n>To tackle the challenge of image synthesis with grounding answers, we propose a rule-to-image synthesis pipeline.<n> Experiments demonstrate that VLM trained using GRPO on VisualSphinx benefit from logical coherence and readability of our dataset.
  arXiv  Detail & Related papers  (2025-05-29T20:08:36Z)
• Symbolic Rule Extraction from Attention-Guided Sparse Representations in Vision Transformers [1.3812010983144802]
  Recent neuro-symbolic approaches have successfully extracted symbolic rule-sets from CNN-based models to enhance interpretability.<n>We propose a framework for symbolic rule extraction from Vision Transformers (ViTs) by introducing a sparse concept layer inspired by Sparse Autoencoders (SAEs)<n>Our method achieves a 5.14% better classification accuracy than the standard ViT while enabling symbolic reasoning.
  arXiv  Detail & Related papers  (2025-05-10T19:45:15Z)
• Visually Descriptive Language Model for Vector Graphics Reasoning [76.42082386029206]
  We propose the Visually Descriptive Language Model (VDLM) to bridge the gap between low-level visual perception and high-level language reasoning.<n>We show that VDLM significantly improves state-of-the-art LMMs like GPT-4o on various multimodal perception and reasoning tasks.
  arXiv  Detail & Related papers  (2024-04-09T17:30:18Z)
• LOGICSEG: Parsing Visual Semantics with Neural Logic Learning and Reasoning [73.98142349171552]
  LOGICSEG is a holistic visual semantic that integrates neural inductive learning and logic reasoning with both rich data and symbolic knowledge. During fuzzy logic-based continuous relaxation, logical formulae are grounded onto data and neural computational graphs, hence enabling logic-induced network training. These designs together make LOGICSEG a general and compact neural-logic machine that is readily integrated into existing segmentation models.
  arXiv  Detail & Related papers  (2023-09-24T05:43:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.