MathWriting: A Dataset For Handwritten Mathematical Expression Recognition

• URL: http://arxiv.org/abs/2404.10690v2
• Date: Sat, 29 Mar 2025 12:18:26 GMT
• Title: MathWriting: A Dataset For Handwritten Mathematical Expression Recognition
• Authors: Philippe Gervais, Anastasiia Fadeeva, Andrii Maksai,
• Abstract summary: MathWriting is the largest online handwritten mathematical expression dataset to date.<n>One MathWriting sample consists of a formula written on a touch screen and a corresponding expression.<n>This dataset can also be used in its rendered form for offline HME recognition.
• Score: 0.9012198585960439
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recognition of handwritten mathematical expressions allows to transfer scientific notes into their digital form. It facilitates the sharing, searching, and preservation of scientific information. We introduce MathWriting, the largest online handwritten mathematical expression dataset to date. It consists of 230k human-written samples and an additional 400k synthetic ones}. This dataset can also be used in its rendered form for offline HME recognition. One MathWriting sample consists of a formula written on a touch screen and a corresponding LaTeX expression. We also provide a normalized version of LaTeX expression to simplify the recognition task and enhance the result quality. We provide baseline performance of standard models like OCR and CTC Transformer as well as Vision-Language Models like PaLI on the dataset. The dataset together with an example colab is accessible on Github.

Related papers

• InfiMM-WebMath-40B: Advancing Multimodal Pre-Training for Enhanced Mathematical Reasoning [58.7966588457529]
  InfiMM-WebMath-40B is a high-quality dataset of interleaved image-text documents. It comprises 24 million web pages, 85 million associated image URLs, and 40 billion text tokens, all meticulously extracted and filtered from CommonCrawl. Our evaluations on text-only benchmarks show that, despite utilizing only 40 billion tokens, our dataset significantly enhances the performance of our 1.3B model. Our models set a new state-of-the-art among open-source models on multi-modal math benchmarks such as MathVerse and We-Math.
  arXiv  Detail & Related papers  (2024-09-19T08:41:21Z)
• TeXBLEU: Automatic Metric for Evaluate LaTeX Format [4.337656290539519]
  We propose BLEU, a metric for evaluating mathematical expressions in the format built on the n-gram-based BLEU metric. The proposed BLEU consists of a tokenizer trained on the arXiv paper dataset and a fine-tuned embedding model with positional encoding.
  arXiv  Detail & Related papers  (2024-09-10T16:54:32Z)
• Handling Numeric Expressions in Automatic Speech Recognition [56.972851337263755]
  We compare cascaded and end-to-end approaches to recognize and format numeric expression. Results show that adapted end-to-end models offer competitive performance with the advantage of lower latency and inference cost.
  arXiv  Detail & Related papers  (2024-07-18T09:46:19Z)
• PosFormer: Recognizing Complex Handwritten Mathematical Expression with Position Forest Transformer [51.260384040953326]
  Handwritten Mathematical Expression Recognition (HMER) has wide applications in human-machine interaction scenarios. We propose a position forest transformer (PosFormer) for HMER, which jointly optimize two tasks: expression recognition and position recognition. PosFormer consistently outperforms the state-of-the-art methods 2.03%/1.22%/2, 1.83%, and 4.62% gains on datasets.
  arXiv  Detail & Related papers  (2024-07-10T15:42:58Z)
• Discovering symbolic expressions with parallelized tree search [59.92040079807524]
  Symbolic regression plays a crucial role in scientific research thanks to its capability of discovering concise and interpretable mathematical expressions from data. Existing algorithms have faced a critical bottleneck of accuracy and efficiency over a decade when handling problems of complexity. We introduce a parallelized tree search (PTS) model to efficiently distill generic mathematical expressions from limited data.
  arXiv  Detail & Related papers  (2024-07-05T10:41:15Z)
• ICAL: Implicit Character-Aided Learning for Enhanced Handwritten Mathematical Expression Recognition [9.389169879626428]
  This paper introduces a novel approach, Implicit Character-Aided Learning (ICAL), to mine the global expression information. By modeling and utilizing implicit character information, ICAL achieves a more accurate and context-aware interpretation of handwritten mathematical expressions.
  arXiv  Detail & Related papers  (2024-05-15T02:03:44Z)
• MathNet: A Data-Centric Approach for Printed Mathematical Expression Recognition [2.325171167252542]
  We present an improved version of the benchmark dataset im2latex-100k, featuring 30 fonts instead of one. Second, we introduce the real-world dataset realFormula, with MEs extracted from papers. Third, we developed a MER model, MathNet, based on a convolutional vision transformer, with superior results on all four test sets.
  arXiv  Detail & Related papers  (2024-04-21T14:03:34Z)
• Improving Text Embeddings with Large Language Models [59.930513259982725]
  We introduce a novel and simple method for obtaining high-quality text embeddings using only synthetic data and less than 1k training steps. We leverage proprietary LLMs to generate diverse synthetic data for hundreds of thousands of text embedding tasks across 93 languages. Experiments demonstrate that our method achieves strong performance on highly competitive text embedding benchmarks without using any labeled data.
  arXiv  Detail & Related papers  (2023-12-31T02:13:18Z)
• Learning Multiplex Representations on Text-Attributed Graphs with One Language Model Encoder [55.24276913049635]
  We propose METAG, a new framework for learning Multiplex rEpresentations on Text-Attributed Graphs. In contrast to existing methods, METAG uses one text encoder to model the shared knowledge across relations. We conduct experiments on nine downstream tasks in five graphs from both academic and e-commerce domains.
  arXiv  Detail & Related papers  (2023-10-10T14:59:22Z)
• WizardMath: Empowering Mathematical Reasoning for Large Language Models via Reinforced Evol-Instruct [128.89645483139236]
  We present WizardMath, which enhances the mathematical reasoning abilities of Llama-2, by applying our proposed Reinforcement Learning from Evol-Instruct Feedback (RLEIF) method to the domain of math. Our model even surpasses ChatGPT-3.5, Claude Instant-1, PaLM-2 and Minerva on GSM8k, simultaneously surpasses Text-davinci, PaLM-1 and GPT-3 on MATH.
  arXiv  Detail & Related papers  (2023-08-18T14:23:21Z)
• Recognizing Handwritten Mathematical Expressions of Vertical Addition and Subtraction [2.945134482768693]
  This work proposes a new handwritten elementary mathematical expression dataset composed of addition and subtraction expressions in a vertical format. We also extended the MNIST dataset to generate artificial images with this structure. Our analysis evaluated the object detection algorithms YOLO v7, YOLO v8, YOLO-NAS, NanoDet and FCOS for identifying the mathematical symbols.
  arXiv  Detail & Related papers  (2023-08-10T18:39:35Z)
• GENIUS: Sketch-based Language Model Pre-training via Extreme and Selective Masking for Text Generation and Augmentation [76.7772833556714]
  We introduce GENIUS: a conditional text generation model using sketches as input. GENIUS is pre-trained on a large-scale textual corpus with a novel reconstruction from sketch objective. We show that GENIUS can be used as a strong and ready-to-use data augmentation tool for various natural language processing (NLP) tasks.
  arXiv  Detail & Related papers  (2022-11-18T16:39:45Z)
• PART: Pre-trained Authorship Representation Transformer [64.78260098263489]
  Authors writing documents imprint identifying information within their texts: vocabulary, registry, punctuation, misspellings, or even emoji usage. Previous works use hand-crafted features or classification tasks to train their authorship models, leading to poor performance on out-of-domain authors. We propose a contrastively trained model fit to learn textbfauthorship embeddings instead of semantics.
  arXiv  Detail & Related papers  (2022-09-30T11:08:39Z)
• Self-Supervised Pretraining of Graph Neural Network for the Retrieval of Related Mathematical Expressions in Scientific Articles [8.942112181408156]
  We propose a new approach for retrieval of mathematical expressions based on machine learning. We design an unsupervised representation learning task that combines embedding learning with self-supervised learning. We collect a huge dataset with over 29 million mathematical expressions from over 900,000 publications published on arXiv.org.
  arXiv  Detail & Related papers  (2022-08-22T12:11:30Z)
• Offline Handwritten Mathematical Recognition using Adversarial Learning and Transformers [3.9220281834178463]
  offline HMER is often viewed as a much harder problem as compared to online HMER. In this paper, we purpose a encoder-decoder model that uses paired adversarial learning. We have been able to improve latest CROHME 2019 test set results by 4% approx.
  arXiv  Detail & Related papers  (2022-08-20T11:45:02Z)
• Syntax-Aware Network for Handwritten Mathematical Expression Recognition [53.130826547287626]
  Handwritten mathematical expression recognition (HMER) is a challenging task that has many potential applications. Recent methods for HMER have achieved outstanding performance with an encoder-decoder architecture. We propose a simple and efficient method for HMER, which is the first to incorporate syntax information into an encoder-decoder network.
  arXiv  Detail & Related papers  (2022-03-03T09:57:19Z)
• Unsupervised Training Data Generation of Handwritten Formulas using Generative Adversarial Networks with Self-Attention [3.785514121306353]
  We introduce a system that creates a large set of synthesized training examples of mathematical expressions which are derived from documents. For this purpose, we propose a novel attention-based generative adversarial network to translate rendered equations to handwritten formulas. The datasets generated by this approach contain hundreds of thousands of formulas, making it ideal for pretraining or the design of more complex models.
  arXiv  Detail & Related papers  (2021-06-17T12:27:18Z)
• Disambiguating Symbolic Expressions in Informal Documents [2.423990103106667]
  We present a dataset with roughly 33,000 entries. We describe a methodology using a transformer language model pre-trained on sources obtained from arxiv.org. We evaluate our model using a plurality of dedicated techniques, taking the syntax and semantics of symbolic expressions into account.
  arXiv  Detail & Related papers  (2021-01-25T10:14:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.