Toward an Artificial General Teacher: Procedural Geometry Data Generation and Visual Grounding with Vision-Language Models

• URL: http://arxiv.org/abs/2604.02893v1
• Date: Fri, 03 Apr 2026 09:10:21 GMT
• Title: Toward an Artificial General Teacher: Procedural Geometry Data Generation and Visual Grounding with Vision-Language Models
• Abstract summary: We study visual explanation in geometry education as a Referring Image (RIS) problem.<n>Given a diagram and a natural language description, the task is to produce a pixel-level mask for the referred geometric element.<n>Existing RIS models trained on natural image benchmarks such as RefCOCO fail catastrophically on geometric diagrams.
• Score: 0.0050417894187191285
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We study visual explanation in geometry education as a Referring Image Segmentation (RIS) problem: given a diagram and a natural language description, the task is to produce a pixel-level mask for the referred geometric element. However, existing RIS models trained on natural image benchmarks such as RefCOCO fail catastrophically on geometric diagrams due to the fundamental domain shift between photographic scenes and abstract, textureless schematics. To address the absence of suitable training data, we present a fully automated procedural data engine that generates over 200,000 synthetic geometry diagrams with pixel-perfect segmentation masks and linguistically diverse referring expressions, requiring zero manual annotation. We further propose domain-specific fine-tuning of vision-language models (VLMs), demonstrating that a fine-tuned Florence-2 achieves 49% IoU and 85% Buffered IoU (BIoU), compared to <1% IoU in zero-shot settings. We introduce Buffered IoU, a geometry-aware evaluation metric that accounts for thin-structure localization, and show that it better reflects true segmentation quality than standard IoU. Our results establish a foundation for building Artificial General Teachers (AGTs) capable of providing visually grounded, step-by-step explanations of geometry problems.

Related papers

• Geoparsing: Diagram Parsing for Plane and Solid Geometry with a Unified Formal Language [60.70116693224378]
  We design a unified formal language that integrates plane and solid geometry, comprehensively covering geometric structures and semantic relations.<n>We construct GDP-29K, a large-scale dataset comprising 20k plane and 9k solid geometry samples collected from diverse real-world sources.
  arXiv  Detail & Related papers  (2026-04-13T15:09:56Z)
• Learning 3D Representations for Spatial Intelligence from Unposed Multi-View Images [81.94999489820974]
  UniSplat is a feed-forward framework for learning 3D representations from unposed multi-view images.<n>We introduce a dual-masking strategy that strengthens geometry induction in the encoder.<n>Second, we develop a coarse-to-fine Gaussian splatting strategy that reduces appearance-semantics inconsistencies.<n>Third, we introduce a pose-conditioned recalibration mechanism that interrelates predicted 3D point and semantic maps into the image plane.
  arXiv  Detail & Related papers  (2026-04-12T10:36:18Z)
• Hierarchical Process Reward Models are Symbolic Vision Learners [56.94353087007494]
  Symbolic computer vision represents diagrams through explicit logical rules and structured representations, enabling interpretable understanding in machine vision.<n>This requires fundamentally different learning paradigms from pixel-based visual models.<n>We propose a novel self-supervised auto-encoder that encodes diagrams into primitives and decodes them through our executable engine to reconstruct input diagrams.
  arXiv  Detail & Related papers  (2025-12-02T18:46:40Z)
• Seeing through Imagination: Learning Scene Geometry via Implicit Spatial World Modeling [68.14113731953971]
  This paper introduces MILO, an Implicit spatIaL wOrld modeling paradigm that simulates human-like imagination.<n>We show that our approach significantly enhances spatial reasoning capabilities across multiple baselines and benchmarks.
  arXiv  Detail & Related papers  (2025-12-01T16:01:41Z)
• GeoMVD: Geometry-Enhanced Multi-View Generation Model Based on Geometric Information Extraction [15.701540201818192]
  Multi-view image generation holds significant application value in computer vision.<n>Existing methods, which rely on extending single images, face notable computational challenges in maintaining cross-view consistency.<n>We propose the Geometry-guided Multi-View Diffusion Model, which incorporates mechanisms for extracting multi-view geometric information.
  arXiv  Detail & Related papers  (2025-11-15T13:17:18Z)
• GeoX: Geometric Problem Solving Through Unified Formalized Vision-Language Pre-training [45.42400674977197]
  GeoX is a multi-modal large model focusing on geometric understanding and reasoning tasks.<n>We introduce unimodal pre-training to develop a diagram encoder and symbol decoder, enhancing the understanding of geometric images and corpora.<n>We propose a Generator-And-Sampler Transformer (GS-Former) to generate discriminative queries and eliminate uninformative representations from unevenly distributed geometric signals.
  arXiv  Detail & Related papers  (2024-12-16T15:20:03Z)
• Disentangling Visual Priors: Unsupervised Learning of Scene Interpretations with Compositional Autoencoder [0.20718016474717196]
  We propose a neurosymbolic architecture that uses a domain-specific language to capture selected priors of image formation. We express template programs in that language and learn their parameterization with features extracted from the scene by a convolutional neural network. When executed, the parameterized program produces geometric primitives which are rendered and assessed for correspondence with the scene content.
  arXiv  Detail & Related papers  (2024-09-15T12:47:39Z)
• Diagram Formalization Enhanced Multi-Modal Geometry Problem Solver [11.69164802295844]
  We introduce a new framework that integrates visual features, geometric formal language, and natural language representations. We propose a novel synthetic data approach and create a large-scale geometric dataset, SynthGeo228K, annotated with both formal and natural language captions. Our framework improves MLLMs' ability to process geometric diagrams and extends their application to open-ended tasks on the formalgeo7k dataset.
  arXiv  Detail & Related papers  (2024-09-06T12:11:06Z)
• Self-Supervised Image Representation Learning with Geometric Set Consistency [50.12720780102395]
  We propose a method for self-supervised image representation learning under the guidance of 3D geometric consistency. Specifically, we introduce 3D geometric consistency into a contrastive learning framework to enforce the feature consistency within image views.
  arXiv  Detail & Related papers  (2022-03-29T08:57:33Z)
• DONet: Learning Category-Level 6D Object Pose and Size Estimation from Depth Observation [53.55300278592281]
  We propose a method of Category-level 6D Object Pose and Size Estimation (COPSE) from a single depth image. Our framework makes inferences based on the rich geometric information of the object in the depth channel alone. Our framework competes with state-of-the-art approaches that require labeled real-world images.
  arXiv  Detail & Related papers  (2021-06-27T10:41:50Z)
• Learning 3D Face Reconstruction with a Pose Guidance Network [49.13404714366933]
  We present a self-supervised learning approach to learning monocular 3D face reconstruction with a pose guidance network (PGN) First, we unveil the bottleneck of pose estimation in prior parametric 3D face learning methods, and propose to utilize 3D face landmarks for estimating pose parameters. With our specially designed PGN, our model can learn from both faces with fully labeled 3D landmarks and unlimited unlabeled in-the-wild face images.
  arXiv  Detail & Related papers  (2020-10-09T06:11:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.