Global Context for improving recognition of Online Handwritten Mathematical Expressions

• URL: http://arxiv.org/abs/2105.10156v1
• Date: Fri, 21 May 2021 06:39:47 GMT
• Title: Global Context for improving recognition of Online Handwritten Mathematical Expressions
• Authors: Cuong Tuan Nguyen, Thanh-Nghia Truong, Hung Tuan Nguyen and Masaki Nakagawa
• Abstract summary: We present a temporal classification method for online handwritten mathematical expressions (HMEs) The method benefits from global context of a deep bidirectional Long Short-term Memory network. To recognize an online HME, a symbol-level parse tree with Context-Free Grammar is constructed.
• Score: 7.868468656324007
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper presents a temporal classification method for all three subtasks of symbol segmentation, symbol recognition and relation classification in online handwritten mathematical expressions (HMEs). The classification model is trained by multiple paths of symbols and spatial relations derived from the Symbol Relation Tree (SRT) representation of HMEs. The method benefits from global context of a deep bidirectional Long Short-term Memory network, which learns the temporal classification directly from online handwriting by the Connectionist Temporal Classification loss. To recognize an online HME, a symbol-level parse tree with Context-Free Grammar is constructed, where symbols and spatial relations are obtained from the temporal classification results. We show the effectiveness of the proposed method on the two latest CROHME datasets.

Related papers

• Local and Global Graph Modeling with Edge-weighted Graph Attention Network for Handwritten Mathematical Expression Recognition [3.419173524128023]
  We introduce an End-to-end model with an Edge-weighted Graph Attention Mechanism (EGAT) to perform simultaneous node and edge classification. We also propose a stroke-level Graph Modeling method for both local (LGM) and global (GGM) information. Our system demonstrates superior performance in symbol detection, relation classification, and expression-level recognition.
  arXiv  Detail & Related papers  (2024-10-24T08:59:27Z)
• Spatial Action Unit Cues for Interpretable Deep Facial Expression Recognition [55.97779732051921]
  State-of-the-art classifiers for facial expression recognition (FER) lack interpretability, an important feature for end-users. A new learning strategy is proposed to explicitly incorporate AU cues into classifier training, allowing to train deep interpretable models. Our new strategy is generic, and can be applied to any deep CNN- or transformer-based classifier without requiring any architectural change or significant additional training time.
  arXiv  Detail & Related papers  (2024-10-01T10:42:55Z)
• Dual Branch Network Towards Accurate Printed Mathematical Expression Recognition [27.428642277844972]
  A Dual Branch transformer-based Network (DBN) is proposed to learn both local and global context information for accurate Printed Mathematical Expression Recognition. Our experimental results have demonstrated that DBN can accurately recognize mathematical expressions and has achieved state-of-the-art performance.
  arXiv  Detail & Related papers  (2023-12-14T15:30:34Z)
• Semantic Graph Representation Learning for Handwritten Mathematical Expression Recognition [57.60390958736775]
  We propose a simple but efficient method to enhance semantic interaction learning (SIL) We first construct a semantic graph based on the statistical symbol co-occurrence probabilities. Then we design a semantic aware module (SAM), which projects the visual and classification feature into semantic space. Our method achieves better recognition performance than prior arts on both CROHME and HME100K datasets.
  arXiv  Detail & Related papers  (2023-08-21T06:23:41Z)
• Direct Embedding of Temporal Network Edges via Time-Decayed Line Graphs [51.51417735550026]
  Methods for machine learning on temporal networks generally exhibit at least one of two limitations. We present a simple method that avoids both shortcomings: construct the line graph of the network, which includes a node for each interaction, and weigh the edges of this graph based on the difference in time between interactions. Empirical results on real-world networks demonstrate our method's efficacy and efficiency on both edge classification and temporal link prediction.
  arXiv  Detail & Related papers  (2022-09-30T18:24:13Z)
• Sign Language Translation with Hierarchical Spatio-TemporalGraph Neural Network [6.623802929157273]
  Sign language translation (SLT) generates text in a spoken language from visual content in a sign language. In this paper, these unique characteristics of sign languages are formulated as hierarchical-temporal graph representations. A novel deep learning architecture, namely hierarchical hierarchical-temporal graph neural network (HSTG-NN), is proposed.
  arXiv  Detail & Related papers  (2021-11-14T07:02:28Z)
• Hierarchical Heterogeneous Graph Representation Learning for Short Text Classification [60.233529926965836]
  We propose a new method called SHINE, which is based on graph neural network (GNN) for short text classification. First, we model the short text dataset as a hierarchical heterogeneous graph consisting of word-level component graphs. Then, we dynamically learn a short document graph that facilitates effective label propagation among similar short texts.
  arXiv  Detail & Related papers  (2021-10-30T05:33:05Z)
• Learning symbol relation tree for online mathematical expression recognition [7.868468656324007]
  This paper proposes a method for recognizing online handwritten mathematical expressions (OnHME) by building a symbol relation tree (SRT) directly from a sequence of strokes. A bidirectional recurrent neural network learns from multiple derived paths of SRT to predict both symbols and spatial relations between symbols using global context. The recognition system achieves 44.12% and 41.76% expression recognition rates on the Competition on Recognition of Online Handwritten Mathematical expressions (CROHME) 2014 and 2016 testing sets.
  arXiv  Detail & Related papers  (2021-05-13T05:18:17Z)
• Learning the Implicit Semantic Representation on Graph-Structured Data [57.670106959061634]
  Existing representation learning methods in graph convolutional networks are mainly designed by describing the neighborhood of each node as a perceptual whole. We propose a Semantic Graph Convolutional Networks (SGCN) that explores the implicit semantics by learning latent semantic-paths in graphs.
  arXiv  Detail & Related papers  (2021-01-16T16:18:43Z)
• Multi-Level Graph Convolutional Network with Automatic Graph Learning for Hyperspectral Image Classification [63.56018768401328]
  We propose a Multi-level Graph Convolutional Network (GCN) with Automatic Graph Learning method (MGCN-AGL) for HSI classification. By employing attention mechanism to characterize the importance among spatially neighboring regions, the most relevant information can be adaptively incorporated to make decisions. Our MGCN-AGL encodes the long range dependencies among image regions based on the expressive representations that have been produced at local level.
  arXiv  Detail & Related papers  (2020-09-19T09:26:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.