HATFormer: Historic Handwritten Arabic Text Recognition with Transformers

• URL: http://arxiv.org/abs/2410.02179v1
• Date: Thu, 3 Oct 2024 03:43:29 GMT
• Title: HATFormer: Historic Handwritten Arabic Text Recognition with Transformers
• Authors: Adrian Chan, Anupam Mijar, Mehreen Saeed, Chau-Wai Wong, Akram Khater,
• Abstract summary: Arabic handwriting datasets are smaller compared to English ones, making it difficult to train generalizable Arabic HTR models. We propose HATFormer, a transformer-based encoder-decoder architecture that builds on a state-of-the-art English HTR model. Our work demonstrates the feasibility of adapting an English HTR method to a low-resource language with complex, language-specific challenges.
• Score: 6.3660090769559945
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Arabic handwritten text recognition (HTR) is challenging, especially for historical texts, due to diverse writing styles and the intrinsic features of Arabic script. Additionally, Arabic handwriting datasets are smaller compared to English ones, making it difficult to train generalizable Arabic HTR models. To address these challenges, we propose HATFormer, a transformer-based encoder-decoder architecture that builds on a state-of-the-art English HTR model. By leveraging the transformer's attention mechanism, HATFormer captures spatial contextual information to address the intrinsic challenges of Arabic script through differentiating cursive characters, decomposing visual representations, and identifying diacritics. Our customization to historical handwritten Arabic includes an image processor for effective ViT information preprocessing, a text tokenizer for compact Arabic text representation, and a training pipeline that accounts for a limited amount of historic Arabic handwriting data. HATFormer achieves a character error rate (CER) of 8.6% on the largest public historical handwritten Arabic dataset, with a 51% improvement over the best baseline in the literature. HATFormer also attains a comparable CER of 4.2% on the largest private non-historical dataset. Our work demonstrates the feasibility of adapting an English HTR method to a low-resource language with complex, language-specific challenges, contributing to advancements in document digitization, information retrieval, and cultural preservation.

Related papers

• Gazelle: An Instruction Dataset for Arabic Writing Assistance [12.798604366250261]
  We present Gazelle, a comprehensive dataset for Arabic writing assistance. We also offer an evaluation framework designed to enhance Arabic writing assistance tools. Our findings underscore the need for continuous model training and dataset enrichment.
  arXiv  Detail & Related papers  (2024-10-23T17:51:58Z)
• Qalam : A Multimodal LLM for Arabic Optical Character and Handwriting Recognition [18.280762424107408]
  This study introduces Qalam, a novel foundation model designed for Arabic OCR and HWR. Our model significantly outperforms existing methods, achieving a Word Error Rate (WER) of just 0.80% in HWR tasks and 1.18% in OCR tasks.
  arXiv  Detail & Related papers  (2024-07-18T14:31:09Z)
• Muharaf: Manuscripts of Handwritten Arabic Dataset for Cursive Text Recognition [5.28595286827031]
  The Manuscripts of Handwritten Arabic(Muharaf) dataset is a machine learning dataset consisting of more than 1,600 historic handwritten page images. This dataset was compiled to advance the state of the art in handwritten text recognition.
  arXiv  Detail & Related papers  (2024-06-13T23:40:34Z)
• MTVQA: Benchmarking Multilingual Text-Centric Visual Question Answering [58.92057773071854]
  We introduce MTVQA, the first benchmark featuring high-quality human expert annotations across 9 diverse languages. MTVQA is the first benchmark featuring high-quality human expert annotations across 9 diverse languages.
  arXiv  Detail & Related papers  (2024-05-20T12:35:01Z)
• AceGPT, Localizing Large Language Models in Arabic [73.39989503874634]
  The paper proposes a comprehensive solution that includes pre-training with Arabic texts, Supervised Fine-Tuning (SFT) utilizing native Arabic instructions, and GPT-4 responses in Arabic. The goal is to cultivate culturally cognizant and value-aligned Arabic LLMs capable of accommodating the diverse, application-specific needs of Arabic-speaking communities.
  arXiv  Detail & Related papers  (2023-09-21T13:20:13Z)
• Code-Switching Text Generation and Injection in Mandarin-English ASR [57.57570417273262]
  We investigate text generation and injection for improving the performance of an industry commonly-used streaming model, Transformer-Transducer (T-T) We first propose a strategy to generate code-switching text data and then investigate injecting generated text into T-T model explicitly by Text-To-Speech (TTS) conversion or implicitly by tying speech and text latent spaces. Experimental results on the T-T model trained with a dataset containing 1,800 hours of real Mandarin-English code-switched speech show that our approaches to inject generated code-switching text significantly boost the performance of T-T models.
  arXiv  Detail & Related papers  (2023-03-20T09:13:27Z)
• Beyond Arabic: Software for Perso-Arabic Script Manipulation [67.31374614549237]
  We provide a set of finite-state transducer (FST) components and corresponding utilities for manipulating the writing systems of languages that use the Perso-Arabic script. The library also provides simple FST-based romanization and transliteration.
  arXiv  Detail & Related papers  (2023-01-26T20:37:03Z)
• Huruf: An Application for Arabic Handwritten Character Recognition Using Deep Learning [0.0]
  We propose a lightweight Convolutional Neural Network-based architecture for recognizing Arabic characters and digits. The proposed pipeline consists of a total of 18 layers containing four layers each for convolution, pooling, batch normalization, dropout, and finally one Global average layer. The proposed model respectively achieved an accuracy of 96.93% and 99.35% which is comparable to the state-of-the-art and makes it a suitable solution for real-life end-level applications.
  arXiv  Detail & Related papers  (2022-12-16T17:39:32Z)
• New Results for the Text Recognition of Arabic Maghrib{\=i} Manuscripts -- Managing an Under-resourced Script [0.0]
  We introduce and assess a new modus operandi for HTR models development and fine-tuning dedicated to the Arabic Maghrib=i scripts. The comparison between several state-of-the-art HTR models demonstrates the relevance of a word-based neural approach specialized for Arabic. Results open new perspectives for Arabic scripts processing and more generally for poorly-endowed languages processing.
  arXiv  Detail & Related papers  (2022-11-29T12:21:41Z)
• The Effect of Normalization for Bi-directional Amharic-English Neural Machine Translation [53.907805815477126]
  This paper presents the first relatively large-scale Amharic-English parallel sentence dataset. We build bi-directional Amharic-English translation models by fine-tuning the existing Facebook M2M100 pre-trained model. The results show that the normalization of Amharic homophone characters increases the performance of Amharic-English machine translation in both directions.
  arXiv  Detail & Related papers  (2022-10-27T07:18:53Z)
• Graphemic Normalization of the Perso-Arabic Script [47.429213930688086]
  This paper documents the challenges that Perso-Arabic presents beyond the best-documented languages. We focus on the situation in natural language processing (NLP), which is affected by multiple, often neglected, issues. We evaluate the effects of script normalization on eight languages from diverse language families in the Perso-Arabic script diaspora on machine translation and statistical language modeling tasks.
  arXiv  Detail & Related papers  (2022-10-21T21:59:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.