Visual Set Program Synthesizer

• URL: http://arxiv.org/abs/2603.15997v1
• Date: Mon, 16 Mar 2026 23:15:54 GMT
• Title: Visual Set Program Synthesizer
• Abstract summary: We propose treating visual reasoning as Visual Program Synthesis, where the model first generates a symbolic program that is executed by a separate engine grounded in visual scenes.<n>We also introduce Set-VQA, a new benchmark designed specifically for evaluating set-based visual reasoning.
• Score: 43.33875468039207
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: A user pointing their phone at a supermarket shelf and asking "Which soda has the least sugar?" poses a difficult challenge for current visual Al assistants. Such queries require not only object recognition, but explicit set-based reasoning such as filtering, comparison, and aggregation. Standard endto-end MLLMs often fail at these tasks because they lack an explicit mechanism for compositional logic. We propose treating visual reasoning as Visual Program Synthesis, where the model first generates a symbolic program that is executed by a separate engine grounded in visual scenes. We also introduce Set-VQA, a new benchmark designed specifically for evaluating set-based visual reasoning. Experiments show that our approach significantly outperforms state-of-the-art baselines on complex reasoning tasks, producing more systematic and transparent behavior while substantially improving answer accuracy. These results demonstrate that program-driven reasoning provides a principled alternative to black-box visual-language inference.

Related papers

• Saliency-Aware Multi-Route Thinking: Revisiting Vision-Language Reasoning [50.62037276161025]
  Vision-language models (VLMs) aim to reason by jointly leveraging visual and textual modalities.<n>Key obstacle is that visual inputs are typically provided only once at the start of generation.<n>We propose emphSaliency-Aware Principle (SAP) selection.
  arXiv  Detail & Related papers  (2026-02-18T18:49:56Z)
• CoT-Seg: Rethinking Segmentation with Chain-of-Thought Reasoning and Self-Correction [50.67483317563736]
  This paper aims to explore a system that can think step-by-step, look up information if needed, generate results, self-evaluate its own results, and refine the results.<n>We introduce CoT-Seg, a training-free framework that rethinks reasoning segmentation by combining chain-of-thought reasoning with self-correction.
  arXiv  Detail & Related papers  (2026-01-24T11:41:54Z)
• RECODE: Reasoning Through Code Generation for Visual Question Answering [68.86938437188964]
  We propose to leverage derendering -- the process of reverse-engineering visuals into executable code -- as a new modality for verifiable visual reasoning.<n>Our work demonstrates that grounding visual perception in executable code provides a new path toward more accurate and verifiable multimodal reasoning.
  arXiv  Detail & Related papers  (2025-10-15T17:05:37Z)
• Visual Programmability: A Guide for Code-as-Thought in Chart Understanding [37.44645754630439]
  We propose a Code-as-Thought (CaT) approach to represent the visual information of a chart in a verifiable, symbolic format.<n>Visual Programmability is a learnable property that determines if a chart-question pair is better solved with code or direct visual analysis.<n>We implement this concept in an adaptive framework where a Vision-Language Models (VLMs) learns to choose between the CaT pathway and a direct visual reasoning pathway.
  arXiv  Detail & Related papers  (2025-09-11T09:22:16Z)
• Multi-Step Visual Reasoning with Visual Tokens Scaling and Verification [22.871255950998016]
  We introduce a novel framework for inference-time visual tokens scaling that enables MLLMs to perform verifier-guided reasoning over visual content.<n>Our method significantly outperforms existing approaches across diverse visual reasoning benchmarks.<n>These results demonstrate the promise of dynamic inference mechanisms for enabling fine-grained, context-aware visual reasoning in next-generation MLLMs.
  arXiv  Detail & Related papers  (2025-06-08T17:38:49Z)
• VisuRiddles: Fine-grained Perception is a Primary Bottleneck for Multimodal Large Language Models in Abstract Visual Reasoning [70.44416154144001]
  Recent strides in multimodal large language models (MLLMs) have significantly advanced their performance in many reasoning tasks.<n> Abstract Visual Reasoning (AVR) remains a critical challenge, primarily due to limitations in perceiving abstract graphics.<n>We propose VisuRiddles, a benchmark for PRS, featuring tasks meticulously constructed to assess models' reasoning capacities.<n>Second, we introduce the Perceptual Riddle Synthesizer (PRS), an automated framework for generating riddles with fine-grained perceptual descriptions.
  arXiv  Detail & Related papers  (2025-06-03T07:24:00Z)
• LongPerceptualThoughts: Distilling System-2 Reasoning for System-1 Perception [105.78609483419115]
  We introduce LongPerceptualThoughts, a new synthetic dataset with 30K long-thought traces for perceptual tasks.<n>We propose a novel three-stage data synthesis framework that first synthesizes verifiable multiple-choice questions.<n>We demonstrate notable improvements over existing visual reasoning data-generation methods.
  arXiv  Detail & Related papers  (2025-04-21T18:10:38Z)
• NAVER: A Neuro-Symbolic Compositional Automaton for Visual Grounding with Explicit Logic Reasoning [22.60247555240363]
  This paper explores challenges for methods that require reasoning like human cognition.<n>We propose NAVER, a compositional visual grounding method that integrates explicit probabilistic logic reasoning.<n>Our results show that NAVER achieves SoTA performance comparing to recent end-to-end and compositional baselines.
  arXiv  Detail & Related papers  (2025-02-01T09:19:08Z)
• SHOP-VRB: A Visual Reasoning Benchmark for Object Perception [26.422761228628698]
  We present an approach and a benchmark for visual reasoning in robotics applications. We focus on inferring object properties from visual and text data. We propose a reasoning system based on symbolic program execution.
  arXiv  Detail & Related papers  (2020-04-06T13:46:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.