Spatial Attention and Syntax Rule Enhanced Tree Decoder for Offine Handwritten Mathematical Expression Recognition

• URL: http://arxiv.org/abs/2303.07077v1
• Date: Mon, 13 Mar 2023 12:59:53 GMT
• Title: Spatial Attention and Syntax Rule Enhanced Tree Decoder for Offine Handwritten Mathematical Expression Recognition
• Authors: Zihao Lin, Jinrong Li, Fan Yang, Shuangping Huang, Xu Yang, Jianmin Lin and Ming Yang
• Abstract summary: We propose a novel model called Spatial Attention and Syntax Rule Enhanced Tree Decoder (SS-TD) Our model can effectively describe tree structure and increase the accuracy of output expression. Experiments show that SS-TD achieves better recognition performance than prior models on CROHME 14/16/19 datasets.
• Score: 12.656673677551778
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Offline Handwritten Mathematical Expression Recognition (HMER) has been dramatically advanced recently by employing tree decoders as part of the encoder-decoder method. Despite the tree decoder-based methods regard the expressions as a tree and parse 2D spatial structure to the tree nodes sequence, the performance of existing works is still poor due to the inevitable tree nodes prediction errors. Besides, they lack syntax rules to regulate the output of expressions. In this paper, we propose a novel model called Spatial Attention and Syntax Rule Enhanced Tree Decoder (SS-TD), which is equipped with spatial attention mechanism to alleviate the prediction error of tree structure and use syntax masks (obtained from the transformation of syntax rules) to constrain the occurrence of ungrammatical mathematical expression. In this way, our model can effectively describe tree structure and increase the accuracy of output expression. Experiments show that SS-TD achieves better recognition performance than prior models on CROHME 14/16/19 datasets, demonstrating the effectiveness of our model.

Related papers

• TAMER: Tree-Aware Transformer for Handwritten Mathematical Expression Recognition [17.855238221599635]
  We propose a novel model named TAMER (Tree-Aware Transformer) for handwritten mathematical expression recognition. TAMER combines the advantages of both sequence decoding and tree decoding models by jointly optimizing sequence prediction and tree structure prediction tasks. Experimental results on CROHME datasets demonstrate that TAMER outperforms traditional sequence decoding models.
  arXiv  Detail & Related papers  (2024-08-16T07:24:19Z)
• A Transformer Architecture for Online Gesture Recognition of Mathematical Expressions [0.0]
  Transformer architecture is shown to provide an end-to-end model for building expression trees from online handwritten gestures corresponding to glyph strokes. The attention mechanism was successfully used to encode, learn and enforce the underlying syntax of expressions. For the first time, the encoder is fed with unseen online-temporal data tokens potentially forming an infinitely large vocabulary.
  arXiv  Detail & Related papers  (2022-11-04T17:55:55Z)
• Structure-Unified M-Tree Coding Solver for MathWord Problem [57.825176412485504]
  In previous work, models designed by taking into account the properties of the binary tree structure of mathematical expressions at the output side have achieved better performance. In this paper, we propose the Structure-Unified M-Tree Coding Coding (S-UMCr), which applies a tree with any M branches (M-tree) to unify the output structures. Experimental results on the widely used MAWPS and Math23K datasets have demonstrated that SUMC-r not only outperforms several state-of-the-art models but also performs much better under low-resource conditions.
  arXiv  Detail & Related papers  (2022-10-22T12:20:36Z)
• Syntax-Aware Network for Handwritten Mathematical Expression Recognition [53.130826547287626]
  Handwritten mathematical expression recognition (HMER) is a challenging task that has many potential applications. Recent methods for HMER have achieved outstanding performance with an encoder-decoder architecture. We propose a simple and efficient method for HMER, which is the first to incorporate syntax information into an encoder-decoder network.
  arXiv  Detail & Related papers  (2022-03-03T09:57:19Z)
• Incorporating Constituent Syntax for Coreference Resolution [50.71868417008133]
  We propose a graph-based method to incorporate constituent syntactic structures. We also explore to utilise higher-order neighbourhood information to encode rich structures in constituent trees. Experiments on the English and Chinese portions of OntoNotes 5.0 benchmark show that our proposed model either beats a strong baseline or achieves new state-of-the-art performance.
  arXiv  Detail & Related papers  (2022-02-22T07:40:42Z)
• A Syntax-Guided Grammatical Error Correction Model with Dependency Tree Correction [83.14159143179269]
  Grammatical Error Correction (GEC) is a task of detecting and correcting grammatical errors in sentences. We propose a syntax-guided GEC model (SG-GEC) which adopts the graph attention mechanism to utilize the syntactic knowledge of dependency trees. We evaluate our model on public benchmarks of GEC task and it achieves competitive results.
  arXiv  Detail & Related papers  (2021-11-05T07:07:48Z)
• Recursive Tree Grammar Autoencoders [3.791857415239352]
  We propose a novel autoencoder approach that encodes trees via a bottom-up grammar and decodes trees via a tree grammar. We show experimentally that our proposed method improves the autoencoding error, training time, and optimization score on four benchmark datasets.
  arXiv  Detail & Related papers  (2020-12-03T17:37:25Z)
• Recursive Top-Down Production for Sentence Generation with Latent Trees [77.56794870399288]
  We model the production property of context-free grammars for natural and synthetic languages. We present a dynamic programming algorithm that marginalises over latent binary tree structures with $N$ leaves. We also present experimental results on German-English translation on the Multi30k dataset.
  arXiv  Detail & Related papers  (2020-10-09T17:47:16Z)
• Transformer-Based Neural Text Generation with Syntactic Guidance [0.0]
  We study the problem of using (partial) constituency parse trees as syntactic guidance for controlled text generation. Our method first expands a partial template parse tree to a full-fledged parse tree tailored for the input source text. Our experiments in the controlled paraphrasing task show that our method outperforms SOTA models both semantically and syntactically.
  arXiv  Detail & Related papers  (2020-10-05T01:33:58Z)
• Tree-structured Attention with Hierarchical Accumulation [103.47584968330325]
  "Hierarchical Accumulation" encodes parse tree structures into self-attention at constant time complexity. Our approach outperforms SOTA methods in four IWSLT translation tasks and the WMT'14 English-German translation task.
  arXiv  Detail & Related papers  (2020-02-19T08:17:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.