Unsupervised Training Data Generation of Handwritten Formulas using Generative Adversarial Networks with Self-Attention

• URL: http://arxiv.org/abs/2106.09432v1
• Date: Thu, 17 Jun 2021 12:27:18 GMT
• Title: Unsupervised Training Data Generation of Handwritten Formulas using Generative Adversarial Networks with Self-Attention
• Authors: Matthias Springstein and Eric M\"uller-Budack and Ralph Ewerth
• Abstract summary: We introduce a system that creates a large set of synthesized training examples of mathematical expressions which are derived from documents. For this purpose, we propose a novel attention-based generative adversarial network to translate rendered equations to handwritten formulas. The datasets generated by this approach contain hundreds of thousands of formulas, making it ideal for pretraining or the design of more complex models.
• Score: 3.785514121306353
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The recognition of handwritten mathematical expressions in images and video frames is a difficult and unsolved problem yet. Deep convectional neural networks are basically a promising approach, but typically require a large amount of labeled training data. However, such a large training dataset does not exist for the task of handwritten formula recognition. In this paper, we introduce a system that creates a large set of synthesized training examples of mathematical expressions which are derived from LaTeX documents. For this purpose, we propose a novel attention-based generative adversarial network to translate rendered equations to handwritten formulas. The datasets generated by this approach contain hundreds of thousands of formulas, making it ideal for pretraining or the design of more complex models. We evaluate our synthesized dataset and the recognition approach on the CROHME 2014 benchmark dataset. Experimental results demonstrate the feasibility of the approach.

Related papers

• Self-Supervised Representation Learning for Online Handwriting Text Classification [0.8594140167290099]
  We propose the novel Part of Stroke Masking (POSM) as a pretext task for pretraining models to extract informative representations from the online handwriting of individuals in English and Chinese languages. To evaluate the quality of the extracted representations, we use both intrinsic and extrinsic evaluation methods. The pretrained models are fine-tuned to achieve state-of-the-art results in tasks such as writer identification, gender classification, and handedness classification.
  arXiv  Detail & Related papers  (2023-10-10T14:07:49Z)
• Syntax-Aware Network for Handwritten Mathematical Expression Recognition [53.130826547287626]
  Handwritten mathematical expression recognition (HMER) is a challenging task that has many potential applications. Recent methods for HMER have achieved outstanding performance with an encoder-decoder architecture. We propose a simple and efficient method for HMER, which is the first to incorporate syntax information into an encoder-decoder network.
  arXiv  Detail & Related papers  (2022-03-03T09:57:19Z)
• Data-to-text Generation with Macro Planning [61.265321323312286]
  We propose a neural model with a macro planning stage followed by a generation stage reminiscent of traditional methods. Our approach outperforms competitive baselines in terms of automatic and human evaluation.
  arXiv  Detail & Related papers  (2021-02-04T16:32:57Z)
• Learning to Segment Human Body Parts with Synthetically Trained Deep Convolutional Networks [58.0240970093372]
  This paper presents a new framework for human body part segmentation based on Deep Convolutional Neural Networks trained using only synthetic data. The proposed approach achieves cutting-edge results without the need of training the models with real annotated data of human body parts.
  arXiv  Detail & Related papers  (2021-02-02T12:26:50Z)
• Disambiguating Symbolic Expressions in Informal Documents [2.423990103106667]
  We present a dataset with roughly 33,000 entries. We describe a methodology using a transformer language model pre-trained on sources obtained from arxiv.org. We evaluate our model using a plurality of dedicated techniques, taking the syntax and semantics of symbolic expressions into account.
  arXiv  Detail & Related papers  (2021-01-25T10:14:37Z)
• Learning the Implicit Semantic Representation on Graph-Structured Data [57.670106959061634]
  Existing representation learning methods in graph convolutional networks are mainly designed by describing the neighborhood of each node as a perceptual whole. We propose a Semantic Graph Convolutional Networks (SGCN) that explores the implicit semantics by learning latent semantic-paths in graphs.
  arXiv  Detail & Related papers  (2021-01-16T16:18:43Z)
• Neural Language Modeling for Contextualized Temporal Graph Generation [49.21890450444187]
  This paper presents the first study on using large-scale pre-trained language models for automated generation of an event-level temporal graph for a document.
  arXiv  Detail & Related papers  (2020-10-20T07:08:00Z)
• Omni-supervised Facial Expression Recognition via Distilled Data [120.11782405714234]
  We propose omni-supervised learning to exploit reliable samples in a large amount of unlabeled data for network training. We experimentally verify that the new dataset can significantly improve the ability of the learned FER model. To tackle this, we propose to apply a dataset distillation strategy to compress the created dataset into several informative class-wise images.
  arXiv  Detail & Related papers  (2020-05-18T09:36:51Z)
• Recognizing Handwritten Mathematical Expressions as LaTex Sequences Using a Multiscale Robust Neural Network [3.9164573079514016]
  A robust multiscale neural network is proposed to recognize handwritten mathematical expressions and output sequences. With the addition of visualization, the model's recognition process is shown in detail. The present model results suggest that the state-of-the-art model has better robustness, fewer errors, and higher accuracy.
  arXiv  Detail & Related papers  (2020-02-26T12:39:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.