Language-Guided Abstraction for Visual Reasoning

• URL: http://arxiv.org/abs/2606.12847v1
• Date: Thu, 11 Jun 2026 03:22:00 GMT
• Title: Language-Guided Abstraction for Visual Reasoning
• Abstract summary: We propose a novel framework that enhances visual reasoning via a language-guided Learning Using Privileged Information branch.<n>Specifically, we design a Semantic Compression Module by feeding a unified, task-agnostic prompt into DeepSeek-V3.<n>We also design a Cross-Attention Projector to align visual features with semantic embeddings, aiming to guide the training of the ARC model.
• Score: 8.097020439992205
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The Abstraction and Reasoning Corpus (ARC) is viewed as a critical avenue to Artificial General Intelligence (AGI), as it enables models to learn abstract transformation rules from few-shot examples and then generalize to new tasks. However, prevalent ARC methodology is either pure language or vision-only (i.e., VARC). The former depends heavily on LLMs, consuming billions of parameters. The latter often struggles to capture high-level semantics, leading to overfitting on pixel-level patterns. To bridge this gap, we propose L-VARC, a novel framework that enhances visual reasoning via a language-guided Learning Using Privileged Information (LUPI) branch. Specifically, we design a Semantic Compression Module by feeding a unified, task-agnostic prompt into DeepSeek-V3. In this way, the raw LARC (a crowd-sourced language description dataset) can be substantially refined and structured, fitting with the context length constraint of standard text encoders (e.g., CLIP). Moreover, we design a Cross-Attention Projector to align visual features with semantic embeddings, aiming to guide the training of the ARC model. Notably, the LUPI branch is taken in the training process and will be discarded during inference, thereby yielding a lightweight model with a mere 18 million parameters. Extensive experiments demonstrate that our L-VARC effectively leverages linguistic priors to boost visual reasoning and outperforms state-of-the-art. Ablation studies further confirm the contribution of the two new designs towards the L-VARC framework. The code is available at https://github.com/GZHU-DVL/L-VARC.

Related papers

• Think Visually, Reason Textually: Vision-Language Synergy in ARC [94.15522924153264]
  ARC-AGI is a rigorous testbed for conceptual rule induction and transfer to novel tasks.<n>Most existing methods treat ARC-AGI as a purely textual reasoning task, overlooking the fact that humans rely heavily on visual abstraction.<n>We introduce two synergistic strategies: Vision-Language Synergy Reasoning (VLSR) and Modality-Switch Self-Correction (MSSC)<n>Our findings suggest that unifying visual abstraction with linguistic reasoning is a crucial step toward achieving generalizable, human-like intelligence.
  arXiv  Detail & Related papers  (2025-11-19T18:59:04Z)
• ARC Is a Vision Problem! [50.59206008530851]
  We formulate ARC within a vision paradigm, framing it as an image-to-image translation problem.<n>Our framework, termed Vision ARC, achieves 60.4% accuracy on the ARC-1 benchmark.
  arXiv  Detail & Related papers  (2025-11-18T18:59:49Z)
• EasyARC: Evaluating Vision Language Models on True Visual Reasoning [0.0]
  We introduce EasyARC, a vision-language benchmark requiring multi-image, multi-step reasoning, and self-correction.<n>EasyARC is procedurally generated, fully verifiable, and scalable, making it ideal for reinforcement learning pipelines.<n>We benchmark state-of-the-art vision-language models and analyze their failure modes.
  arXiv  Detail & Related papers  (2025-06-13T09:03:33Z)
• Tackling the Abstraction and Reasoning Corpus with Vision Transformers: the Importance of 2D Representation, Positions, and Objects [31.926206783846144]
  We show that a Vision Transformer (ViT) fails dramatically on most ARC tasks even when trained on one million examples per task.<n>We propose ViTARC, a ViT-style architecture that unlocks some of the visual reasoning capabilities required by the ARC.<n>Our task-specific ViTARC models achieve a test solve rate close to 100% on more than half of the 400 public ARC tasks.
  arXiv  Detail & Related papers  (2024-10-08T22:25:34Z)
• Towards Learning Abductive Reasoning using VSA Distributed Representations [56.31867341825068]
  We introduce the Abductive Rule Learner with Context-awareness (ARLC) model. ARLC features a novel and more broadly applicable training objective for abductive reasoning. We show ARLC's robustness to post-programming training by incrementally learning from examples on top of programmed knowledge.
  arXiv  Detail & Related papers  (2024-06-27T12:05:55Z)
• Vision-by-Language for Training-Free Compositional Image Retrieval [78.60509831598745]
  Compositional Image Retrieval (CIR) aims to retrieve the relevant target image in a database. Recent research sidesteps this need by using large-scale vision-language models (VLMs) We propose to tackle CIR in a training-free manner via Vision-by-Language (CIReVL)
  arXiv  Detail & Related papers  (2023-10-13T17:59:38Z)
• In-context Autoencoder for Context Compression in a Large Language Model [70.7621953091318]
  We propose the In-context Autoencoder (ICAE) to compress a long context into short compact memory slots. ICAE is first pretrained using both autoencoding and language modeling objectives on massive text data.
  arXiv  Detail & Related papers  (2023-07-13T17:59:21Z)
• LLMs and the Abstraction and Reasoning Corpus: Successes, Failures, and the Importance of Object-based Representations [50.431003245201644]
  We show that GPT-4 is unable to "reason" perfectly within non-language domains such as the 1D-ARC or a simple ARC subset. We propose an object-based representation that is obtained through an external tool, resulting in nearly doubling the performance on solved ARC tasks and near-perfect scores on the easier 1D-ARC.
  arXiv  Detail & Related papers  (2023-05-26T16:32:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.