A Multi-Modal Neural Geometric Solver with Textual Clauses Parsed from Diagram

• URL: http://arxiv.org/abs/2302.11097v2
• Date: Fri, 28 Apr 2023 10:04:17 GMT
• Title: A Multi-Modal Neural Geometric Solver with Textual Clauses Parsed from Diagram
• Authors: Ming-Liang Zhang, Fei Yin, Cheng-Lin Liu
• Abstract summary: We propose a new neural solver called PGPSNet to fuse multi-modal information efficiently. PGPSNet is endowed with rich knowledge of geometry theorems and geometric representation. We build a new large-scale and fine-annotated GPS dataset named PGPS9K.
• Score: 33.62866585222121
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Geometry problem solving (GPS) is a high-level mathematical reasoning requiring the capacities of multi-modal fusion and geometric knowledge application. Recently, neural solvers have shown great potential in GPS but still be short in diagram presentation and modal fusion. In this work, we convert diagrams into basic textual clauses to describe diagram features effectively, and propose a new neural solver called PGPSNet to fuse multi-modal information efficiently. Combining structural and semantic pre-training, data augmentation and self-limited decoding, PGPSNet is endowed with rich knowledge of geometry theorems and geometric representation, and therefore promotes geometric understanding and reasoning. In addition, to facilitate the research of GPS, we build a new large-scale and fine-annotated GPS dataset named PGPS9K, labeled with both fine-grained diagram annotation and interpretable solution program. Experiments on PGPS9K and an existing dataset Geometry3K validate the superiority of our method over the state-of-the-art neural solvers. Our code, dataset and appendix material are available at \url{https://github.com/mingliangzhang2018/PGPS}.

Related papers

• Fuse, Reason and Verify: Geometry Problem Solving with Parsed Clauses from Diagram [78.79651421493058]
  We propose a neural-symbolic model for plane geometry problem solving (PGPS) with three key steps: modal fusion, reasoning process and knowledge verification. For reasoning, we design an explicable solution program to describe the geometric reasoning process, and employ a self-limited decoder to generate solution program autoregressively. We also construct a large-scale geometry problem dataset called PGPS9K, containing fine-grained annotations of textual clauses, solution program and involved knowledge solvers.
  arXiv  Detail & Related papers  (2024-07-10T02:45:22Z)
• A Survey of Geometric Graph Neural Networks: Data Structures, Models and Applications [67.33002207179923]
  This paper presents a survey of data structures, models, and applications related to geometric GNNs. We provide a unified view of existing models from the geometric message passing perspective. We also summarize the applications as well as the related datasets to facilitate later research for methodology development and experimental evaluation.
  arXiv  Detail & Related papers  (2024-03-01T12:13:04Z)
• G-LLaVA: Solving Geometric Problem with Multi-Modal Large Language Model [124.68242155098189]
  Large language models (LLMs) have shown remarkable proficiency in human-level reasoning and generation capabilities. G-LLaVA demonstrates exceptional performance in solving geometric problems, significantly outperforming GPT-4-V on the MathVista benchmark with only 7B parameters.
  arXiv  Detail & Related papers  (2023-12-18T17:36:20Z)
• Simplicial Representation Learning with Neural $k$-Forms [14.566552361705499]
  This paper focuses on leveraging geometric information from simplicial complexes embedded in $mathbbRn$ using node coordinates. We use differential k-forms in mathbbRn to create representations of simplices, offering interpretability and geometric consistency without message passing. Our method is efficient, versatile, and applicable to various input complexes, including graphs, simplicial complexes, and cell complexes.
  arXiv  Detail & Related papers  (2023-12-13T21:03:39Z)
• LANS: A Layout-Aware Neural Solver for Plane Geometry Problem [43.192629815250285]
  We propose a layout-aware neural solver named LANS, integrated with two new modules: multimodal layout-aware pre-trained language module and layout-aware fusion attention (LA-FA) Experiments on datasets Geometry3K and PGPS9K validate the effectiveness of the layout-aware modules and superior problem-solving performance of our LANS solver, over existing symbolic and neural solvers.
  arXiv  Detail & Related papers  (2023-11-25T04:11:19Z)
• GeoQA: A Geometric Question Answering Benchmark Towards Multimodal Numerical Reasoning [172.36214872466707]
  We focus on solving geometric problems, which requires a comprehensive understanding of textual descriptions, visual diagrams, and theorem knowledge. We propose a Geometric Question Answering dataset GeoQA, containing 5,010 geometric problems with corresponding annotated programs.
  arXiv  Detail & Related papers  (2021-05-30T12:34:17Z)
• Inter-GPS: Interpretable Geometry Problem Solving with Formal Language and Symbolic Reasoning [123.06420835072225]
  We construct a new large-scale benchmark, Geometry3K, consisting of 3,002 geometry problems with dense annotation in formal language. We propose a novel geometry solving approach with formal language and symbolic reasoning, called Interpretable Geometry Problem solver (Inter-GPS) Inter-GPS incorporates theorem knowledge as conditional rules and performs symbolic reasoning step by step.
  arXiv  Detail & Related papers  (2021-05-10T07:46:55Z)
• Fusing the Old with the New: Learning Relative Camera Pose with Geometry-Guided Uncertainty [91.0564497403256]
  We present a novel framework that involves probabilistic fusion between the two families of predictions during network training. Our network features a self-attention graph neural network, which drives the learning by enforcing strong interactions between different correspondences. We propose motion parmeterizations suitable for learning and show that our method achieves state-of-the-art performance on the challenging DeMoN and ScanNet datasets.
  arXiv  Detail & Related papers  (2021-04-16T17:59:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.