A Benchmark of State-Space Models vs. Transformers and BiLSTM-based Models for Historical Newspaper OCR

• URL: http://arxiv.org/abs/2604.00725v1
• Date: Wed, 01 Apr 2026 10:33:33 GMT
• Title: A Benchmark of State-Space Models vs. Transformers and BiLSTM-based Models for Historical Newspaper OCR
• Abstract summary: Linear-time State-Space Models (SSMs) are a scalable alternative to Transformer-based sequence modeling for OCR.<n>We present the first OCR architecture based on SSMs, combining a CNN visual encoder with bi-directional and autoregressive Mamba sequence modeling.<n>We show that Mamba-based models maintain competitive accuracy while halving inference time and exhibiting superior memory scaling.
• Score: 4.247423625723055
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: End-to-end OCR for historical newspapers remains challenging, as models must handle long text sequences, degraded print quality, and complex layouts. While Transformer-based recognizers dominate current research, their quadratic complexity limits efficient paragraph-level transcription and large-scale deployment. We investigate linear-time State-Space Models (SSMs), specifically Mamba, as a scalable alternative to Transformer-based sequence modeling for OCR. We present to our knowledge, the first OCR architecture based on SSMs, combining a CNN visual encoder with bi-directional and autoregressive Mamba sequence modeling, and conduct a large-scale benchmark comparing SSMs with Transformer- and BiLSTM-based recognizers. Multiple decoding strategies (CTC, autoregressive, and non-autoregressive) are evaluated under identical training conditions alongside strong neural baselines (VAN, DAN, DANIEL) and widely used off-the-shelf OCR engines (PERO-OCR, Tesseract OCR, TrOCR, Gemini). Experiments on historical newspapers from the Bibliothèque nationale du Luxembourg, with newly released >99% verified gold-standard annotations, and cross-dataset tests on Fraktur and Antiqua lines, show that all neural models achieve low error rates (~2% CER), making computational efficiency the main differentiator. Mamba-based models maintain competitive accuracy while halving inference time and exhibiting superior memory scaling (1.26x vs 2.30x growth at 1000 chars), reaching 6.07% CER at the severely degraded paragraph level compared to 5.24% for DAN, while remaining 2.05x faster. We release code, trained models, and standardized evaluation protocols to enable reproducible research and guide practitioners in large-scale cultural heritage OCR.

Related papers

• TongGuOCR: A Layout-Aware and Token-Augmented OCR MLLM for Chinese Historical Documents [50.64307290680222]
  TongGuOCR is a layout-aware and token-augmented multimodal large language model (MLLM) for OCR of Chinese historical documents.<n>TongGuOCR achieves 93.76 AR and reduces NED from 10.43 to 6.15 and RO-ED from 7.53 to 3.49 relative to the best competing score for each metric.
  arXiv  Detail & Related papers  (2026-08-08T04:50:00Z)
• Scaling State-Space Models from Lines to Paragraphs: An Ablation of Mamba-based OCR [4.247423625723054]
  State-Space Models (SSMs) offer linear-time decoding and have been shown to match Transformer accuracy on printed historical lines.<n>We study how a Mamba-based OCR recognizer scales from lines to paragraphs.<n>Our study clarifies when SSMs are a practical choice for large-scale document transcription and when they are not.
  arXiv  Detail & Related papers  (2026-06-22T16:07:40Z)
• PP-OCRv5: A Specialized 5M-Parameter Model Rivaling Billion-Parameter Vision-Language Models on OCR Tasks [21.41974664575541]
  "OCR 2.0" and large-scale vision-language models (VLMs) have set new benchmarks in text recognition.<n>This paper introduces PP-OCRv5, a meticulously optimized, lightweight OCR system with merely 5 million parameters.
  arXiv  Detail & Related papers  (2026-03-25T14:54:40Z)
• GLM-OCR Technical Report [65.42028025507491]
  GLM-OCR is an efficient compact model designed for real-world document understanding.<n>It combines a CogViT visual encoder with a GLM language decoder, achieving a strong balance between computational efficiency and recognition performance.<n>Extensive evaluations on public benchmarks and industrial scenarios show that GLM-OCR achieves competitive or state-of-the-art performance.
  arXiv  Detail & Related papers  (2026-03-11T15:55:47Z)
• LINA: Linear Autoregressive Image Generative Models with Continuous Tokens [56.80443965097921]
  Autoregressive models with continuous tokens form a promising paradigm for visual generation, especially for text-to-image (T2I) synthesis.<n>We study how to design compute-efficient linear attention within this framework.<n>We present LINA, a simple and compute-efficient T2I model built entirely on linear attention, capable of generating high-fidelity 1024x1024 images from user instructions.
  arXiv  Detail & Related papers  (2026-01-30T06:44:33Z)
• Datarus-R1: An Adaptive Multi-Step Reasoning LLM for Automated Data Analysis [0.0]
  We present Datarus-R1-14B, a language model fine-tuned from Qwen 2.5-14B-Instruct to act as a virtual data analyst and graduate-level problem solver.<n>Datarus is trained not on isolated question-answer pairs but on full analytical trajectories including reasoning steps, code execution, error traces, self-corrections, and final conclusions.
  arXiv  Detail & Related papers  (2025-08-18T21:58:18Z)
• Coupling Speech Encoders with Downstream Text Models [4.679869237248675]
  We present a modular approach to building cascade speech translation models. We preserve state-of-the-art speech recognition (ASR) and text translation (MT) performance for a given task.
  arXiv  Detail & Related papers  (2024-07-24T19:29:13Z)
• Benchmarking Neural Decoding Backbones towards Enhanced On-edge iBCI Applications [28.482461973598593]
  This study seeks to identify an optimal neural decoding backbone that boasts robust performance and swift inference capabilities suitable for edge deployment. We evaluated four prospective models, Gated Recurrent Unit (GRU), Transformer, Receptance Weighted Key Value (RWKV), and Selective State Space model (Mamba) The findings indicate that although the GRU model delivers sufficient accuracy, the RWKV and Mamba models are preferable due to their superior inference and calibration speeds.
  arXiv  Detail & Related papers  (2024-06-08T02:45:36Z)
• The Languini Kitchen: Enabling Language Modelling Research at Different Scales of Compute [66.84421705029624]
  We introduce an experimental protocol that enables model comparisons based on equivalent compute, measured in accelerator hours. We pre-process an existing large, diverse, and high-quality dataset of books that surpasses existing academic benchmarks in quality, diversity, and document length. This work also provides two baseline models: a feed-forward model derived from the GPT-2 architecture and a recurrent model in the form of a novel LSTM with ten-fold throughput.
  arXiv  Detail & Related papers  (2023-09-20T10:31:17Z)
• Score-based Generative Modeling in Latent Space [93.8985523558869]
  Score-based generative models (SGMs) have recently demonstrated impressive results in terms of both sample quality and distribution coverage. Here, we propose the Latent Score-based Generative Model (LSGM), a novel approach that trains SGMs in a latent space. Moving from data to latent space allows us to train more expressive generative models, apply SGMs to non-continuous data, and learn smoother SGMs in a smaller space.
  arXiv  Detail & Related papers  (2021-06-10T17:26:35Z)
• TextGNN: Improving Text Encoder via Graph Neural Network in Sponsored Search [11.203006652211075]
  We propose a TextGNN model that naturally extends the strong twin tower structured encoders with the complementary graph information from user historical behaviors. In offline experiments, the model achieves a 0.14% overall increase in ROC-AUC with a 1% increased accuracy for long-tail low-frequency Ads. In online A/B testing, the model shows a 2.03% increase in Revenue Per Mille with a 2.32% decrease in Ad defect rate.
  arXiv  Detail & Related papers  (2021-01-15T23:12:47Z)
• Towards a Competitive End-to-End Speech Recognition for CHiME-6 Dinner Party Transcription [73.66530509749305]
  In this paper, we argue that, even in difficult cases, some end-to-end approaches show performance close to the hybrid baseline. We experimentally compare and analyze CTC-Attention versus RNN-Transducer approaches along with RNN versus Transformer architectures. Our best end-to-end model based on RNN-Transducer, together with improved beam search, reaches quality by only 3.8% WER abs. worse than the LF-MMI TDNN-F CHiME-6 Challenge baseline.
  arXiv  Detail & Related papers  (2020-04-22T19:08:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.