Thinking with Constructions: A Benchmark and Policy Optimization for Visual-Text Interleaved Geometric Reasoning

• URL: http://arxiv.org/abs/2603.18662v1
• Date: Thu, 19 Mar 2026 09:27:20 GMT
• Title: Thinking with Constructions: A Benchmark and Policy Optimization for Visual-Text Interleaved Geometric Reasoning
• Abstract summary: We present a framework for Visual-Text Interleaved Chain-of-Thought.<n>We first introduce GeoAux-Bench, the first benchmark comprising 4,334 geometry problems that align textual construction steps with ground-truth visual updates.<n>Our pilot study reveals two critical insights: (1) interleaved visual-textual aids outperform single-modality counterparts, which cannot losslessly capture geometric synergy; and (2) valid constructions act as entropy reducers, strongly correlating with reduced reasoning perplexity.
• Score: 45.18040184559187
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Geometric reasoning inherently requires "thinking with constructions" -- the dynamic manipulation of visual aids to bridge the gap between problem conditions and solutions. However, existing Multimodal Large Language Models (MLLMs) are largely confined to passive inference with static diagrams, lacking the strategic knowledge of when and how to construct effective visual aids. To address this, we present a framework for Visual-Text Interleaved Chain-of-Thought. We first introduce GeoAux-Bench, the first benchmark comprising 4,334 geometry problems that aligns textual construction steps with ground-truth visual updates. Our pilot study reveals two critical insights: (1) interleaved visual-textual aids outperform single-modality counterparts, which cannot losslessly capture geometric synergy; and (2) valid constructions act as entropy reducers, strongly correlating with reduced reasoning perplexity. Building on these findings, we propose Action Applicability Policy Optimization (A2PO), a reinforcement learning paradigm for mastering strategic construction. A2PO employs Adaptive Reward Shaping to regulate the timing and quality of visual aids via counterfactual sampling to distinguish necessary from redundant constructions. Experiments demonstrate our approach enables MLLMs to leverage selective auxiliary constructions, yielding a 3.51% gain over strong baselines. Code and data are available on GitHub.

Related papers

• Think with Structured Grounding: Perceptual Reinforcement Learning for Chart and Visual-Tabular Understanding [41.288665175678794]
  Think with Structured Grounding (TwSG) is a novel fine-grained image perception framework designed to internalize complex images's tool-use capabilities within the model.<n>We use an MLLM to identify key regions guided by ground-truth answers, and then prompt a teacher model to generate high-quality visual question-answering data.<n>Our training pipeline consists of two stages: (1) a cold-start supervised fine-tuning (SFT) phase using multi-turn data with focused area descriptions to foster complex reasoning and error recovery; and (2) a reinforcement fine-tuning phase driven by a novel process reward mechanism
  arXiv  Detail & Related papers  (2026-08-23T14:07:14Z)
• EGL-SCA: Structural Credit Assignment for Co-Evolving Instructions and Tools in Graph Reasoning Agents [35.25814217014108]
  We propose EGL-SCA, a verifier-centric dual-space framework for graph reasoning agents.<n>We show that EGL-SCA achieves a state-of-the-art 92.0% average success rate.
  arXiv  Detail & Related papers  (2026-05-11T11:09:32Z)
• LatentGeo: Learnable Auxiliary Constructions in Latent Space for Multimodal Geometric Reasoning [32.39048489202347]
  We propose a framework that learns continuous latent visual representations to internalize auxiliary geometric constructions without pixel-level rendering or external executors.<n>LatentGeo achieves substantial gains on geometric reasoning tasks, particularly those requiring auxiliary constructions.
  arXiv  Detail & Related papers  (2026-03-12T17:01:23Z)
• GeoVLMath: Enhancing Geometry Reasoning in Vision-Language Models via Cross-Modal Reward for Auxiliary Line Creation [54.53486231309254]
  We present GeoVLMath, an open-source LVLM tailored to auxiliary-line reasoning in solid geometry.<n>We generate textual descriptions of auxiliary-line constructions to better align with the representational strengths of LVLMs.<n>Built on this reward, we present GeoVLMath, an open-source LVLM tailored to auxiliary-line reasoning in solid geometry.
  arXiv  Detail & Related papers  (2025-10-13T05:33:51Z)
• Light-SQ: Structure-aware Shape Abstraction with Superquadrics for Generated Meshes [60.92139345612904]
  We present Light-SQ, a novel superquadric-based optimization framework.<n>We propose a block-regrow-fill strategy guided by structure-aware volumetric decomposition.<n>Experiments demonstrate that Light-SQ enables efficient, high-fidelity, and editable shape abstraction with superquadrics.
  arXiv  Detail & Related papers  (2025-09-29T16:18:32Z)
• GeoSketch: A Neural-Symbolic Approach to Geometric Multimodal Reasoning with Auxiliary Line Construction and Affine Transformation [28.500787311066563]
  GeoSketch is a neural-symbolic framework that recasts geometric reasoning as an interactive perception-reasoning-action loop.<n>By unifying hierarchical decision-making, executable visual actions, and symbolic verification, GeoSketch advances multimodal reasoning from static interpretation to dynamic interaction.
  arXiv  Detail & Related papers  (2025-09-26T15:12:04Z)
• Learning Efficient and Generalizable Graph Retriever for Knowledge-Graph Question Answering [75.12322966980003]
  Large Language Models (LLMs) have shown strong inductive reasoning ability across various domains.<n>Most existing RAG pipelines rely on unstructured text, limiting interpretability and structured reasoning.<n>Recent studies have explored integrating knowledge graphs with LLMs for knowledge graph question answering.<n>We propose RAPL, a novel framework for efficient and effective graph retrieval in KGQA.
  arXiv  Detail & Related papers  (2025-06-11T12:03:52Z)
• GeometryZero: Improving Geometry Solving for LLM with Group Contrastive Policy Optimization [63.107398132743825]
  Group Contrastive Policy Optimization (GCPO) is a novel reinforcement learning framework featuring two key innovations.<n>We develop GeometryZero, a family of affordable-size geometric reasoning models that judiciously determine when to employ auxiliary construction.
  arXiv  Detail & Related papers  (2025-06-08T14:18:15Z)
• A Framework for Reducing the Complexity of Geometric Vision Problems and its Application to Two-View Triangulation with Approximation Bounds [14.419727000332717]
  Triangulation is the task of estimating a 3D point from noisy 2D projections across multiple images.<n>We present a new framework for reducing the computational complexity of geometric vision problems through targeted reweighting of the cost functions used to minimize reprojection errors.<n>Although this work focuses on two-view triangulation, the framework generalizes to other geometric vision problems.
  arXiv  Detail & Related papers  (2025-03-11T08:00:51Z)
• Understanding and Constructing Latent Modality Structures in Multi-modal Representation Learning [53.68371566336254]
  We argue that the key to better performance lies in meaningful latent modality structures instead of perfect modality alignment. Specifically, we design 1) a deep feature separation loss for intra-modality regularization; 2) a Brownian-bridge loss for inter-modality regularization; and 3) a geometric consistency loss for both intra- and inter-modality regularization.
  arXiv  Detail & Related papers  (2023-03-10T14:38:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.