TAMER: Tree-Aware Transformer for Handwritten Mathematical Expression Recognition

• URL: http://arxiv.org/abs/2408.08578v1
• Date: Fri, 16 Aug 2024 07:24:19 GMT
• Title: TAMER: Tree-Aware Transformer for Handwritten Mathematical Expression Recognition
• Authors: Jianhua Zhu, Wenqi Zhao, Yu Li, Xingjian Hu, Liangcai Gao,
• Abstract summary: We propose a novel model named TAMER (Tree-Aware Transformer) for handwritten mathematical expression recognition. TAMER combines the advantages of both sequence decoding and tree decoding models by jointly optimizing sequence prediction and tree structure prediction tasks. Experimental results on CROHME datasets demonstrate that TAMER outperforms traditional sequence decoding models.
• Score: 17.855238221599635
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Handwritten Mathematical Expression Recognition (HMER) has extensive applications in automated grading and office automation. However, existing sequence-based decoding methods, which directly predict $\LaTeX$ sequences, struggle to understand and model the inherent tree structure of $\LaTeX$ and often fail to ensure syntactic correctness in the decoded results. To address these challenges, we propose a novel model named TAMER (Tree-Aware Transformer) for handwritten mathematical expression recognition. TAMER introduces an innovative Tree-aware Module while maintaining the flexibility and efficient training of Transformer. TAMER combines the advantages of both sequence decoding and tree decoding models by jointly optimizing sequence prediction and tree structure prediction tasks, which enhances the model's understanding and generalization of complex mathematical expression structures. During inference, TAMER employs a Tree Structure Prediction Scoring Mechanism to improve the structural validity of the generated $\LaTeX$ sequences. Experimental results on CROHME datasets demonstrate that TAMER outperforms traditional sequence decoding and tree decoding models, especially in handling complex mathematical structures, achieving state-of-the-art (SOTA) performance.

Related papers

• Differentiable Tree Operations Promote Compositional Generalization [106.59434079287661]
  Differentiable Tree Machine (DTM) architecture integrates interpreter with external memory and agent that learns to sequentially select tree operations. DTM achieves 100% while existing baselines such as Transformer, Tree Transformer, LSTM, and Tree2Tree LSTM achieve less than 30%.
  arXiv  Detail & Related papers  (2023-06-01T14:46:34Z)
• Spatial Attention and Syntax Rule Enhanced Tree Decoder for Offine Handwritten Mathematical Expression Recognition [12.656673677551778]
  We propose a novel model called Spatial Attention and Syntax Rule Enhanced Tree Decoder (SS-TD) Our model can effectively describe tree structure and increase the accuracy of output expression. Experiments show that SS-TD achieves better recognition performance than prior models on CROHME 14/16/19 datasets.
  arXiv  Detail & Related papers  (2023-03-13T12:59:53Z)
• Structure-Unified M-Tree Coding Solver for MathWord Problem [57.825176412485504]
  In previous work, models designed by taking into account the properties of the binary tree structure of mathematical expressions at the output side have achieved better performance. In this paper, we propose the Structure-Unified M-Tree Coding Coding (S-UMCr), which applies a tree with any M branches (M-tree) to unify the output structures. Experimental results on the widely used MAWPS and Math23K datasets have demonstrated that SUMC-r not only outperforms several state-of-the-art models but also performs much better under low-resource conditions.
  arXiv  Detail & Related papers  (2022-10-22T12:20:36Z)
• Syntax-Aware Network for Handwritten Mathematical Expression Recognition [53.130826547287626]
  Handwritten mathematical expression recognition (HMER) is a challenging task that has many potential applications. Recent methods for HMER have achieved outstanding performance with an encoder-decoder architecture. We propose a simple and efficient method for HMER, which is the first to incorporate syntax information into an encoder-decoder network.
  arXiv  Detail & Related papers  (2022-03-03T09:57:19Z)
• Semantic Parsing in Task-Oriented Dialog with Recursive Insertion-based Encoder [6.507504084891086]
  We introduce a Recursive INsertion-based entity recognition (RINE) approach for semantic parsing in task-oriented dialog. RINE achieves state-of-the-art exact match accuracy on low- and high-resource versions of the conversational semantic parsing benchmark TOP. Our approach is 2-3.5 times faster than the sequence-to-sequence model at inference time.
  arXiv  Detail & Related papers  (2021-09-09T18:23:45Z)
• SIT3: Code Summarization with Structure-Induced Transformer [48.000063280183376]
  We propose a novel model based on structure-induced self-attention, which encodes sequential inputs with highly-effective structure modeling. Our newly-proposed model achieves new state-of-the-art results on popular benchmarks.
  arXiv  Detail & Related papers  (2020-12-29T11:37:43Z)
• Recursive Top-Down Production for Sentence Generation with Latent Trees [77.56794870399288]
  We model the production property of context-free grammars for natural and synthetic languages. We present a dynamic programming algorithm that marginalises over latent binary tree structures with $N$ leaves. We also present experimental results on German-English translation on the Multi30k dataset.
  arXiv  Detail & Related papers  (2020-10-09T17:47:16Z)
• DRTS Parsing with Structure-Aware Encoding and Decoding [28.711318411470497]
  State-of-the-art performance can be achieved by a neural sequence-to-sequence model. We propose a structural-aware model at both the encoder and decoder phase to integrate the structural information.
  arXiv  Detail & Related papers  (2020-05-14T12:09:23Z)
• Tree-structured Attention with Hierarchical Accumulation [103.47584968330325]
  "Hierarchical Accumulation" encodes parse tree structures into self-attention at constant time complexity. Our approach outperforms SOTA methods in four IWSLT translation tasks and the WMT'14 English-German translation task.
  arXiv  Detail & Related papers  (2020-02-19T08:17:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.