GeoBuildBench: A Benchmark for Interactive and Executable Geometry Construction from Natural Language

• URL: http://arxiv.org/abs/2605.13167v1
• Date: Wed, 13 May 2026 08:30:12 GMT
• Title: GeoBuildBench: A Benchmark for Interactive and Executable Geometry Construction from Natural Language
• Abstract summary: GeoBuildBench is a benchmark designed to evaluate whether large language models and multimodal agents can ground informal natural-language plane geometry problems into executable geometric constructions.<n>We evaluate several state-of-the-art multimodal models in a bounded iterative setting and show that, despite reasonable success rates, models frequently exhibit structural hallucinations, missing objects, and failures to satisfy geometric constraints.
• Score: 18.57905469277832
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We introduce GeoBuildBench, a benchmark designed to evaluate whether large language models and multimodal agents can ground informal natural-language plane geometry problems into executable geometric constructions. Unlike existing geometry benchmarks that focus on answer correctness or static diagram interpretation, GeoBuildBench treats geometry diagram as an interactive construction task: given a textual problem, an agent must generate a domain-specific language (DSL) program to produce a diagram satisfying explicitly specified geometric objects and verifiable constraints. The benchmark features 489 Chinese textbook-style problems, curated through automated filtering and human validation to ensure text-complete, constructible problem specifications. We evaluate several state-of-the-art multimodal models in a bounded iterative setting and show that, despite reasonable success rates, models frequently exhibit structural hallucinations, missing objects, and failures to satisfy geometric constraints, with limited ability to exploit visual and constraint-based feedback for self-correction. These results highlight geometry construction as a rigorous testbed for grounded, executable reasoning beyond textual or visual plausibility. Our benchmark and code are publicly available.

Related papers

• Draw2Think: Harnessing Geometry Reasoning through Constraint Engine Interaction [59.47026381165585]
  Vision-language models solve geometry problems with rising accuracy, yet their intermediate states remain latent and unverifiable.<n>We present Draw2Think, a framework that recasts geometric reasoning from latent spatial inference into agentic interaction with the GeoGebra constraint engine.
  arXiv  Detail & Related papers  (2026-05-20T05:46:14Z)
• Geoparsing: Diagram Parsing for Plane and Solid Geometry with a Unified Formal Language [60.70116693224378]
  We design a unified formal language that integrates plane and solid geometry, comprehensively covering geometric structures and semantic relations.<n>We construct GDP-29K, a large-scale dataset comprising 20k plane and 9k solid geometry samples collected from diverse real-world sources.
  arXiv  Detail & Related papers  (2026-04-13T15:09:56Z)
• Learning Hierarchical and Geometry-Aware Graph Representations for Text-to-CAD [22.163726324892256]
  We propose a hierarchical and geometry-aware graph as an intermediate representation.<n>Instead of mapping text directly to code, our framework first predicts structure and constraints, then conditions action sequencing and code generation.<n>Our method consistently outperforms existing approaches in both geometric fidelity and accurate satisfaction of geometric constraints.
  arXiv  Detail & Related papers  (2026-04-11T07:49:16Z)
• Synthesizing Multimodal Geometry Datasets from Scratch and Enabling Visual Alignment via Plotting Code [27.26235987246201]
  Multimodal geometry reasoning requires models to jointly understand visual diagrams and perform structured symbolic inference.<n>We propose a pipeline for complex multimodal geometry problems from scratch and construct a dataset named textbfGeoCode, which decouples problem generation into symbolic seed construction.<n>We further introduce code prediction as an explicit alignment objective, transforming visual understanding into a supervised structured prediction task.
  arXiv  Detail & Related papers  (2026-02-21T07:53:48Z)
• TangramPuzzle: Evaluating Multimodal Large Language Models with Compositional Spatial Reasoning [104.66714520975837]
  We introduce a geometry-grounded benchmark designed to evaluate compositional spatial reasoning through the lens of the classic Tangram game.<n>We propose the Tangram Construction Expression (TCE), a symbolic geometric framework that grounds tangram assemblies in exact, machine-verifiable coordinate specifications.<n>We conduct extensive evaluation experiments on advanced open-source and proprietary models, revealing an interesting insight: MLLMs tend to prioritize matching the target silhouette while neglecting geometric constraints.
  arXiv  Detail & Related papers  (2026-01-23T07:35:05Z)
• GeoLoom: High-quality Geometric Diagram Generation from Textual Input [41.9055060542649]
  We propose GeoLoom, a novel framework for text-to-diagram generation in geometric domains.<n>GeoLoom comprises two core components: an autoformalization module that translates natural language into a generation-oriented formal language GeoLingua.<n>To support this framework, we introduce GeoNF, a dataset aligning natural language geometric descriptions with formal GeoLingua descriptions.
  arXiv  Detail & Related papers  (2025-12-09T02:22:23Z)
• GeoRef: Referring Expressions in Geometry via Task Formulation, Synthetic Supervision, and Reinforced MLLM-based Solutions [45.70578816057097]
  We introduce the task of Referring Expression (REC) for geometric problems.<n>REC evaluates whether models can localize points, shapes, and spatial relations in diagrams in response to textual prompts.<n>We generate a large-scale synthetic training dataset using a structured geometric formal language.
  arXiv  Detail & Related papers  (2025-09-25T12:00:52Z)
• GeoX: Geometric Problem Solving Through Unified Formalized Vision-Language Pre-training [45.42400674977197]
  GeoX is a multi-modal large model focusing on geometric understanding and reasoning tasks.<n>We introduce unimodal pre-training to develop a diagram encoder and symbol decoder, enhancing the understanding of geometric images and corpora.<n>We propose a Generator-And-Sampler Transformer (GS-Former) to generate discriminative queries and eliminate uninformative representations from unevenly distributed geometric signals.
  arXiv  Detail & Related papers  (2024-12-16T15:20:03Z)
• Adaptive Surface Normal Constraint for Geometric Estimation from Monocular Images [56.86175251327466]
  We introduce a novel approach to learn geometries such as depth and surface normal from images while incorporating geometric context. Our approach extracts geometric context that encodes the geometric variations present in the input image and correlates depth estimation with geometric constraints. Our method unifies depth and surface normal estimations within a cohesive framework, which enables the generation of high-quality 3D geometry from images.
  arXiv  Detail & Related papers  (2024-02-08T17:57:59Z)
• G-LLaVA: Solving Geometric Problem with Multi-Modal Large Language Model [121.07873620883322]
  Large language models (LLMs) have shown remarkable proficiency in human-level reasoning and generation capabilities.<n>G-LLaVA demonstrates exceptional performance in solving geometric problems, significantly outperforming GPT-4-V on the MathVista benchmark with only 7B parameters.
  arXiv  Detail & Related papers  (2023-12-18T17:36:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.