Arabic-Nougat: Fine-Tuning Vision Transformers for Arabic OCR and Markdown Extraction

• URL: http://arxiv.org/abs/2411.17835v1
• Date: Tue, 19 Nov 2024 12:09:12 GMT
• Title: Arabic-Nougat: Fine-Tuning Vision Transformers for Arabic OCR and Markdown Extraction
• Abstract summary: We present Arabic-Nougat, a suite of OCR models for converting Arabic book pages into structured text.<n>Based on Meta's Nougat architecture, Arabic-Nougat includes three specialized models: arabic-small-nougat, arabic-base-nougat, and arabic-large-nougat.<n>Key contributions include the Aranizer-PBE-86k tokenizer, designed for efficient tokenization, and the use of torch.bfloat16 precision with Flash Attention 2.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: We present Arabic-Nougat, a suite of OCR models for converting Arabic book pages into structured Markdown text. Based on Meta's Nougat architecture, Arabic-Nougat includes three specialized models: arabic-small-nougat, arabic-base-nougat, and arabic-large-nougat. These models are fine-tuned on a synthetic dataset, arabic-img2md, comprising 13.7k pairs of Arabic book pages and their Markdown representations. Key contributions include the Aranizer-PBE-86k tokenizer, designed for efficient tokenization, and the use of torch.bfloat16 precision with Flash Attention 2 for optimized training and inference. Our models achieve state-of-the-art performance, with arabic-large-nougat delivering the highest Markdown Structure Accuracy and the lowest Character Error Rate. Additionally, we release a large-scale dataset containing 1.1 billion Arabic tokens extracted from over 8,500 books using our best-performing model, providing a valuable resource for Arabic OCR research. All models, datasets, and code are open-sourced and available at https://github.com/MohamedAliRashad/arabic-nougat.

Related papers

• RightNow-Arabic-0.5B-Turbo: An Open Sub-1B Arabic Language Model via Vocabulary Injection and Edge-First Deployment [0.0]
  We present RightNow-Arabic-0.5B-Turbo, a 518M- parameter Arabic-specialized decoder built on Qwen2.5-0.5B.<n>The pipeline adds 27,032 Arabic tokens via mean-subtoken.<n>The merged model reaches 35.9% mean accuracy, beats every same-class open model, and recovers 67% of SILMA-9B's mean at 1/18 parameters.
  arXiv  Detail & Related papers  (2026-04-10T00:56:50Z)
• Hala Technical Report: Building Arabic-Centric Instruction & Translation Models at Scale [51.41777906371754]
  We present Hala, a family of Arabic-centric instruction and translation models built with our translate-and-tune pipeline.<n>A lightweight language model LFM2-1.2B is then fine-tuned on this data and used to translate high-quality English instruction sets into Arabic.<n>We train Hala models at 350M, 700M, 1.2B, and 9B parameters, and apply slerp merging to balance Arabic specialization with base-model strengths.
  arXiv  Detail & Related papers  (2025-09-17T14:19:28Z)
• mmBERT: A Modern Multilingual Encoder with Annealed Language Learning [57.58071656545661]
  mmBERT is an encoder-only language model pretrained on 3T tokens of multilingual text.<n>We add over 1700 low-resource languages to the data mix only during the decay phase.<n>We show that mmBERT significantly outperforms the previous generation of models on classification and retrieval tasks.
  arXiv  Detail & Related papers  (2025-09-08T17:08:42Z)
• QARI-OCR: High-Fidelity Arabic Text Recognition through Multimodal Large Language Model Adaptation [0.8944616102795021]
  We present Qari-OCR, a vision-language models progressively optimized for Arabic.<n>Qari-OCR establishes a new open-source state-of-the-art with a Word Error Rate (WER) of 0.160, Character Error Rate (CER) of 0.061, and BLEU score of 0.737 on diacritically-rich texts.
  arXiv  Detail & Related papers  (2025-06-02T22:21:06Z)
• Advancing Arabic Reverse Dictionary Systems: A Transformer-Based Approach with Dataset Construction Guidelines [0.8944616102795021]
  This study addresses the critical gap in Arabic natural language processing by developing an effective Arabic Reverse Dictionary (RD) system. We present a novel transformer-based approach with a semi-encoder neural network architecture featuring geometrically decreasing layers. Our methodology incorporates a comprehensive dataset construction process and establishes formal quality standards for Arabic lexicographic definitions.
  arXiv  Detail & Related papers  (2025-04-30T09:56:36Z)
• AIN: The Arabic INclusive Large Multimodal Model [71.29419186696138]
  AIN is an English-Arabic bilingual LMM designed to excel in English and Arabic. AIN demonstrates state-of-the-art Arabic performance, while also possessing strong English-language visual capabilities. AIN's superior capabilities position it as a significant step toward empowering Arabic speakers with advanced multimodal generative AI tools.
  arXiv  Detail & Related papers  (2025-01-31T18:58:20Z)
• Second Language (Arabic) Acquisition of LLMs via Progressive Vocabulary Expansion [55.27025066199226]
  This paper addresses the need for democratizing large language models (LLM) in the Arab world. One practical objective for an Arabic LLM is to utilize an Arabic-specific vocabulary for the tokenizer that could speed up decoding. Inspired by the vocabulary learning during Second Language (Arabic) Acquisition for humans, the released AraLLaMA employs progressive vocabulary expansion.
  arXiv  Detail & Related papers  (2024-12-16T19:29:06Z)
• Swan and ArabicMTEB: Dialect-Aware, Arabic-Centric, Cross-Lingual, and Cross-Cultural Embedding Models and Benchmarks [17.5987429821102]
  Swan is a family of embedding models centred around the Arabic language. Two variants: Swan-Small, based on ARBERTv2, and Swan-Large, built on ArMistral, a pretrained Arabic large language model.
  arXiv  Detail & Related papers  (2024-11-02T09:39:49Z)
• ALLaM: Large Language Models for Arabic and English [9.881560166505452]
  We present ALLaM: Arabic Large Language Model, a series of large language models to support the ecosystem of Arabic Language Technologies (ALT) Our autoregressive decoder-only architecture models demonstrate how second-language acquisition via vocabulary expansion and pretraining can steer a model towards a new language (Arabic) without any catastrophic forgetting in the original language (English) We show that extensive alignment with human preferences can significantly enhance the performance of a language model compared to models of a larger scale with lower quality alignment.
  arXiv  Detail & Related papers  (2024-07-22T05:35:17Z)
• Qalam : A Multimodal LLM for Arabic Optical Character and Handwriting Recognition [18.280762424107408]
  This study introduces Qalam, a novel foundation model designed for Arabic OCR and HWR. Our model significantly outperforms existing methods, achieving a Word Error Rate (WER) of just 0.80% in HWR tasks and 1.18% in OCR tasks.
  arXiv  Detail & Related papers  (2024-07-18T14:31:09Z)
• AlcLaM: Arabic Dialectal Language Model [2.8477895544986955]
  We construct an Arabic dialectal corpus comprising 3.4M sentences gathered from social media platforms. We utilize this corpus to expand the vocabulary and retrain a BERT-based model from scratch. Named AlcLaM, our model was trained using only 13 GB of text, which represents a fraction of the data used by existing models.
  arXiv  Detail & Related papers  (2024-07-18T02:13:50Z)
• GemmAr: Enhancing LLMs Through Arabic Instruction-Tuning [0.0]
  We introduce InstAr-500k, a new Arabic instruction dataset created by generating and collecting content. We assess this dataset by fine-tuning an open-source Gemma-7B model on several downstream tasks to improve its functionality. Based on multiple evaluations, our fine-tuned model achieves excellent performance on several Arabic NLP benchmarks.
  arXiv  Detail & Related papers  (2024-07-02T10:43:49Z)
• On the importance of Data Scale in Pretraining Arabic Language Models [46.431706010614334]
  We conduct a comprehensive study on the role of data in Arabic Pretrained Language Models (PLMs) We reassess the performance of a suite of state-of-the-art Arabic PLMs by retraining them on massive-scale, high-quality Arabic corpora. Our analysis strongly suggests that pretraining data by far is the primary contributor to performance, surpassing other factors.
  arXiv  Detail & Related papers  (2024-01-15T15:11:15Z)
• AceGPT, Localizing Large Language Models in Arabic [73.39989503874634]
  The paper proposes a comprehensive solution that includes pre-training with Arabic texts, Supervised Fine-Tuning (SFT) utilizing native Arabic instructions, and GPT-4 responses in Arabic. The goal is to cultivate culturally cognizant and value-aligned Arabic LLMs capable of accommodating the diverse, application-specific needs of Arabic-speaking communities.
  arXiv  Detail & Related papers  (2023-09-21T13:20:13Z)
• Jais and Jais-chat: Arabic-Centric Foundation and Instruction-Tuned Open Generative Large Language Models [57.76998376458017]
  We introduce Jais and Jais-chat, new state-of-the-art Arabic-centric foundation and instruction-tuned open generative large language models (LLMs) The models are based on the GPT-3 decoder-only architecture and are pretrained on a mixture of Arabic and English texts. We provide a detailed description of the training, the tuning, the safety alignment, and the evaluation of the models.
  arXiv  Detail & Related papers  (2023-08-30T17:07:17Z)
• New Arabic Medical Dataset for Diseases Classification [55.41644538483948]
  We introduce a new Arab medical dataset, which includes two thousand medical documents collected from several Arabic medical websites. The dataset was built for the task of classifying texts and includes 10 classes (Blood, Bone, Cardiovascular, Ear, Endocrine, Eye, Gastrointestinal, Immune, Liver and Nephrological) Experiments on the dataset were performed by fine-tuning three pre-trained models: BERT from Google, Arabert that based on BERT with large Arabic corpus, and AraBioNER that based on Arabert with Arabic medical corpus.
  arXiv  Detail & Related papers  (2021-06-29T10:42:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.