Adapting TrOCR for Printed Tigrinya Text Recognition: Word-Aware Loss Weighting for Cross-Script Transfer Learning

• URL: http://arxiv.org/abs/2604.20813v1
• Date: Wed, 22 Apr 2026 17:43:50 GMT
• Title: Adapting TrOCR for Printed Tigrinya Text Recognition: Word-Aware Loss Weighting for Cross-Script Transfer Learning
• Abstract summary: We present the first adaptation of TrOCR for printed Tigrinya using the Ge'ez script.<n>Starting from a pre-trained model, we extend the byte-level BPE tokenizer to cover 230 Ge'ez characters.<n>All code, model weights, and evaluation scripts are publicly released.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Transformer-based OCR models have shown strong performance on Latin and CJK scripts, but their application to African syllabic writing systems remains limited. We present the first adaptation of TrOCR for printed Tigrinya using the Ge'ez script. Starting from a pre-trained model, we extend the byte-level BPE tokenizer to cover 230 Ge'ez characters and introduce Word-Aware Loss Weighting to resolve systematic word-boundary failures that arise when applying Latin-centric BPE conventions to a new script. The unmodified model produces no usable output on Ge'ez text. After adaptation, the TrOCR-Printed variant achieves 0.22% Character Error Rate and 97.20% exact match accuracy on a held-out test set of 5,000 synthetic images from the GLOCR dataset. An ablation study confirms that Word-Aware Loss Weighting is the critical component, reducing CER by two orders of magnitude compared to vocabulary extension alone. The full pipeline trains in under three hours on a single 8 GB consumer GPU. All code, model weights, and evaluation scripts are publicly released.

Related papers

• When Simpler Is Better: Evaluating Translation Pipelines for Medieval Latin Manuscripts [3.3982074915384874]
  We evaluate the full image-to-translation pipeline on medieval Latin manuscripts.<n> domain-specific Optical Character Recognition (OCR) models reduce character error rate by up to 4.3$times$ compared to general-purpose VLMs.<n>These findings offer both a new benchmark and practical guidance for deploying translation systems in low-resource historical settings.
  arXiv  Detail & Related papers  (2026-07-04T11:58:52Z)
• SN-WER: Script-Normalized WER for Multi-Script Indic ASR Evaluation [0.8243795440723053]
  Script-Normalized WER (SN-WER) is a training-free, evaluation-only scoring method.<n>We evaluate SN-WER on 5 Indic languages, 2 datasets, and 3 ASR models.
  arXiv  Detail & Related papers  (2026-06-01T17:49:10Z)
• GlotOCR Bench: OCR Models Still Struggle Beyond a Handful of Unicode Scripts [58.92151016423978]
  We introduce GlotOCR Bench, a comprehensive benchmark evaluating OCR across 100+ Unicode scripts.<n>Our benchmark comprises clean and degraded image variants rendered from real multilingual texts.<n>We evaluate a broad suite of open-weight and proprietary vision-language models and find that most perform well on fewer than ten scripts.
  arXiv  Detail & Related papers  (2026-04-14T17:12:41Z)
• Global Context Compression with Interleaved Vision-Text Transformation [12.971394377165767]
  In this paper, we investigate global context compression, which saves tokens at both prefilling and inference stages.<n>We propose VIST2, a novel Transformer that interleaves input text chunks alongside their visual encoding.<n>With a 4$times$ compression ratio, the resulting models demonstrate significant superiority over baselines on long writing tasks.
  arXiv  Detail & Related papers  (2026-01-15T13:29:16Z)
• PreP-OCR: A Complete Pipeline for Document Image Restoration and Enhanced OCR Accuracy [14.50674472785442]
  PreP-OCR is a two-stage pipeline that combines document image restoration with semantic-aware post-OCR correction.<n>We show that PreP-OCR reduces character error rates by 63.9-70.3% compared to OCR on raw images.
  arXiv  Detail & Related papers  (2025-05-26T18:25:28Z)
• RETTA: Retrieval-Enhanced Test-Time Adaptation for Zero-Shot Video Captioning [77.59074909960913]
  We propose a novel zero-shot video captioning framework named Retrieval-Enhanced Test-Time Adaptation (RETTA)<n>We bridge video and text using four key models: a general video-text retrieval model XCLIP, a general image-text matching model CLIP, a text alignment model AnglE, and a text generation model GPT-2.<n>To address this problem, we propose using learnable tokens as a communication medium among these four frozen models GPT-2, XCLIP, CLIP, and AnglE.
  arXiv  Detail & Related papers  (2024-05-11T16:22:00Z)
• Data Generation for Post-OCR correction of Cyrillic handwriting [41.94295877935867]
  This paper focuses on the development and application of a synthetic handwriting generation engine based on B'ezier curves. Such an engine generates highly realistic handwritten text in any amounts, which we utilize to create a substantial dataset. We apply a Handwritten Text Recognition (HTR) model to this dataset to identify OCR errors, forming the basis for our POC model training.
  arXiv  Detail & Related papers  (2023-11-27T15:01:26Z)
• ALIP: Adaptive Language-Image Pre-training with Synthetic Caption [78.93535202851278]
  Contrastive Language-Image Pre-training (CLIP) has significantly boosted the performance of various vision-language tasks. The presence of intrinsic noise and unmatched image-text pairs in web data can potentially affect the performance of representation learning. We propose an Adaptive Language-Image Pre-training (ALIP), a bi-path model that integrates supervision from both raw text and synthetic caption.
  arXiv  Detail & Related papers  (2023-08-16T15:19:52Z)
• Cleansing Jewel: A Neural Spelling Correction Model Built On Google OCR-ed Tibetan Manuscripts [12.346821696831805]
  We present a neural spelling correction model built on Google OCR-ed Tibetan Manuscripts to auto-correct OCR-ed noisy output. This paper is divided into four sections: dataset, model architecture, training and analysis.
  arXiv  Detail & Related papers  (2023-04-07T00:45:12Z)
• Code-Switching Text Generation and Injection in Mandarin-English ASR [57.57570417273262]
  We investigate text generation and injection for improving the performance of an industry commonly-used streaming model, Transformer-Transducer (T-T) We first propose a strategy to generate code-switching text data and then investigate injecting generated text into T-T model explicitly by Text-To-Speech (TTS) conversion or implicitly by tying speech and text latent spaces. Experimental results on the T-T model trained with a dataset containing 1,800 hours of real Mandarin-English code-switched speech show that our approaches to inject generated code-switching text significantly boost the performance of T-T models.
  arXiv  Detail & Related papers  (2023-03-20T09:13:27Z)
• LexLIP: Lexicon-Bottlenecked Language-Image Pre-Training for Large-Scale Image-Text Retrieval [71.01982683581572]
  The conventional dense retrieval paradigm relies on encoding images and texts into dense representations using dual-stream encoders. We propose the lexicon-weighting paradigm, where sparse representations in vocabulary space are learned for images and texts. We introduce a novel pre-training framework, that learns importance-aware lexicon representations. Our framework achieves a 5.5 221.3X faster retrieval speed and 13.2 48.8X less index storage memory.
  arXiv  Detail & Related papers  (2023-02-06T16:24:41Z)
• Lexically Aware Semi-Supervised Learning for OCR Post-Correction [90.54336622024299]
  Much of the existing linguistic data in many languages of the world is locked away in non-digitized books and documents. Previous work has demonstrated the utility of neural post-correction methods on recognition of less-well-resourced languages. We present a semi-supervised learning method that makes it possible to utilize raw images to improve performance.
  arXiv  Detail & Related papers  (2021-11-04T04:39:02Z)
• Scarecrow: A Framework for Scrutinizing Machine Text [69.26985439191151]
  We introduce a new structured, crowdsourced error annotation schema called Scarecrow. Scarecrow collects 13k annotations of 1.3k human and machine generate paragraphs of English language news text. These findings demonstrate the value of Scarecrow annotations in the assessment of current and future text generation systems.
  arXiv  Detail & Related papers  (2021-07-02T22:37:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.