DiffMath: Symbol- and Graph-Aware Latent Diffusion Transformer for Handwritten Mathematical Expression Generation

• URL: http://arxiv.org/abs/2606.19939v1
• Date: Thu, 18 Jun 2026 08:37:02 GMT
• Title: DiffMath: Symbol- and Graph-Aware Latent Diffusion Transformer for Handwritten Mathematical Expression Generation
• Abstract summary: Handwritten Mathematical Expression Generation (HMEG) is challenging due to the complex two-dimensional layouts and long-range dependencies.<n>In this work, we propose DiffMath, a symbol-oriented graph-aware latent diffusion framework.<n> Experiments show that DiffMath produces structurally consistent handwritten expressions, superior performance over existing methods, and improves the accuracy of downstream models.
• Score: 47.67907704295109
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Handwritten Mathematical Expression Generation (HMEG) is challenging due to the complex two-dimensional layouts and long-range structural dependencies of mathematical expressions. Existing methods typically rely on explicit spatial supervision, such as symbol-level bounding boxes, which incurs high annotation costs and limits scalability. In this work, we propose DiffMath, a symbol- and graph-aware latent diffusion framework that leverages the hierarchical structure inherent in LaTeX as a structural prior, eliminating the need for positional supervision. First, we design a Relational Abstract Syntax Tree (RelAST), a generation-oriented representation that distills MathML trees into compact triplet sequences [S, R, D], where each token directly encodes a symbol identity, spatial relation, or nesting depth. Second, we introduce MathVAE, which learns structure-preserving latent representations through symbol-aware and relation-aware perceptual regularization, ensuring that the latent space captures both character semantics and spatial topology. Third, MathDiT performs conditional denoising in this structured latent space, further guided by a global symbol-count prior via Adaptive Layer Normalization (AdaLN) to improve structural coherence. Experiments show that DiffMath produces structurally consistent handwritten expressions, achieves superior performance over existing methods, and improves the accuracy of downstream OCR models through synthetic data augmentation.

Related papers

• Weave of Formal Thought [51.56484100374058]
  We introduce Weave of Formal Thought (WoFT), a paradigm uniting rigorous syntactic validation with learned structural representations.<n>We present a latent-variable fine-tuning method training the language model to interleave non-terminal grammar symbols directly into generation.<n>For Python, fine-tuning StarCoder2-3B with our RWS objective reduces per-token cross-entropy by 14.3% relative to a text-only SFT baseline.
  arXiv  Detail & Related papers  (2026-06-24T15:58:11Z)
• From Extrinsic to Intrinsic: Geodesic-Guided Representation Learning for 3D Geometric Data [56.41479248637621]
  We introduce a novel 3D representation learning paradigm, namely textbfPRISM, for textbfPre-training.<n>PRISM learns embeddings by textbfRecovering the textbfIntrinsic textbfSurface geodesic textbfMetric.<n> Experiments demonstrate that our approach shows satisfactory accuracy, robustness, and high efficiency in geodesic distance prediction.
  arXiv  Detail & Related papers  (2026-06-01T13:54:26Z)
• TERGAD: Structure-Aware Text-Enhanced Representations for Graph Anomaly Detection [15.051477921035264]
  Graph Anomaly Detection (GAD) aims to identify atypical graph entities, such as nodes, edges, or substructures, that deviate significantly from the majority.<n>Existing text-rich approaches typically integrate structural context into the data representation pipeline using raw textual features.<n>We propose TERGAD, a novel data augmentation framework that enriches structural semantics for GAD via the semantic reasoning capabilities of Large Language Models.
  arXiv  Detail & Related papers  (2026-05-19T12:09:36Z)
• $\mathcal{S}^2$IT: Stepwise Syntax Integration Tuning for Large Language Models in Aspect Sentiment Quad Prediction [51.165432266846096]
  syntactic structure information is underutilized in the generative paradigm of large language models (LLMs)<n>S2IT progressively integrates syntactic structure knowledge into LLMs through a multi-step tuning process.<n> Experiments demonstrate that S2IT significantly improves state-of-the-art performance across multiple datasets.
  arXiv  Detail & Related papers  (2026-04-25T13:24:29Z)
• Overcoming the Impedance Mismatch: A Theoretical Roadmap for Fusing Foundation Models and Knowledge Graphs [0.0]
  We formalize the underlying structural and geometric friction as the textitImpedance Mismatch.<n>We propose a rigorous theoretical roadmap to achieve true semantic fusion.<n>This framework paves the way for models that seamlessly fuse the precision of symbolic logic with the expressivity of parametric memory.
  arXiv  Detail & Related papers  (2026-04-21T17:30:15Z)
• NAG: A Unified Native Architecture for Encoder-free Text-Graph Modeling in Language Models [33.49410203951687]
  We argue this approach is suboptimal for text-graphs.<n>NAG (Native Architecture for Graphs) is a unified framework that internalizes graph processing within the Language Models.<n>NAG achieves robust graph comprehension without the overhead of external encoders.
  arXiv  Detail & Related papers  (2026-01-30T07:22:11Z)
• PosFormer: Recognizing Complex Handwritten Mathematical Expression with Position Forest Transformer [51.260384040953326]
  Handwritten Mathematical Expression Recognition (HMER) has wide applications in human-machine interaction scenarios. We propose a position forest transformer (PosFormer) for HMER, which jointly optimize two tasks: expression recognition and position recognition. PosFormer consistently outperforms the state-of-the-art methods 2.03%/1.22%/2, 1.83%, and 4.62% gains on datasets.
  arXiv  Detail & Related papers  (2024-07-10T15:42:58Z)
• Frame Averaging for Equivariant Shape Space Learning [85.42901997467754]
  A natural way to incorporate symmetries in shape space learning is to ask that the mapping to the shape space (encoder) and mapping from the shape space (decoder) are equivariant to the relevant symmetries. We present a framework for incorporating equivariance in encoders and decoders by introducing two contributions.
  arXiv  Detail & Related papers  (2021-12-03T06:41:19Z)
• A Study of Continuous Vector Representationsfor Theorem Proving [2.0518509649405106]
  We develop an encoding that allows for logical properties to be preserved and is additionally reversible. This means that the tree shape of a formula including all symbols can be reconstructed from the dense vector representation. We propose datasets that can be used to train these syntactic and semantic properties.
  arXiv  Detail & Related papers  (2021-01-22T15:04:54Z)
• Structure-Augmented Text Representation Learning for Efficient Knowledge Graph Completion [53.31911669146451]
  Human-curated knowledge graphs provide critical supportive information to various natural language processing tasks. These graphs are usually incomplete, urging auto-completion of them. graph embedding approaches, e.g., TransE, learn structured knowledge via representing graph elements into dense embeddings. textual encoding approaches, e.g., KG-BERT, resort to graph triple's text and triple-level contextualized representations.
  arXiv  Detail & Related papers  (2020-04-30T13:50:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.