Page image classifier fine-tuned on century-spanning archives of scanned documents for further content-specific processing

• URL: http://arxiv.org/abs/2606.07558v1
• Date: Mon, 25 May 2026 08:36:36 GMT
• Title: Page image classifier fine-tuned on century-spanning archives of scanned documents for further content-specific processing
• Abstract summary: Digitization projects in the humanities produce vast, heterogeneous archives of historical documents.<n>This work addresses the need for an automated system to classify scanned page images based on visual content type.<n>An image classification system was developed and evaluated on a dataset of over 48,000 annotated historical page images.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Purpose: Digitization projects in the humanities produce vast, heterogeneous archives of historical documents, making manual sorting impractical at scale. This work addresses the need for an automated system to classify scanned page images based on visual content type - text, tables, and graphics - enabling content-specific downstream processing such as Optical Character Recognition (OCR) or structured data extraction. Methods: An image classification system was developed and evaluated on a dataset of over 48,000 annotated historical page images from century-old Czech archaeological archives, refined through four successive annotation stages with domain-expert review. A Random Forest Classifier baseline was established using hand-crafted image features. Subsequently, deep learning architectures were fine-tuned and compared: Convolutional Neural Networks (EfficientNetV2, RegNetY), Vision and Document Image Transformers (ViT, DiT), and multimodal CLIP models. An 11-category label scheme was designed collaboratively with domain experts and evaluated via five-fold cross-validation. Results: The feature-based baseline achieved approximately 75% accuracy. Fine-tuned CNNs and Transformers substantially outperformed it, with RegNetY-16GF achieving 99.16% and ViT-large 99.12% Top-1 accuracy on the held-out test set. CLIP ViT-B/16 reached 99.14% with optimized text descriptions. Conclusion: Image-only models, particularly RegNetY-16GF, deliver near-perfect classification accuracy and produce consistent labels across 649,508 unlabeled archival pages with over 90% inter-model agreement. Fine-tuned CLIP, despite competitive test-set accuracy, showed under 65% agreement with image-only models on unlabeled data, making it less suitable for deployment. The final models, annotated dataset, and software are publicly available under open-source licenses.

Related papers

• MonkeyOCRv2: A Visual-Text Foundation Model for Document AI [71.06391669976786]
  MonkeyOCRv2 is a visual-text pretrained model for document AI.<n> MonkeyDoc v2 is the largest document-image pretraining corpus, comprising 113 million images spanning 17 languages.
  arXiv  Detail & Related papers  (2026-07-13T13:43:39Z)
• Comparative Evaluation of Deep Learning Models for Fake Image Detection [0.0]
  This study compares the performance of four CNN architectures for fake image detection using a unified preprocessing and training pipeline.<n>VGG16 achieved the highest accuracy at 91%, with XceptionNet, ResNet50, and EfficientNetB0 each reaching 90%.<n>The study underscores the need for balanced datasets, advanced augmentation, and fairness-aware training to develop reliable fake image detection systems.
  arXiv  Detail & Related papers  (2026-05-20T09:57:58Z)
• DocIQ: A Benchmark Dataset and Feature Fusion Network for Document Image Quality Assessment [78.21680156380705]
  We introduce a subjective DIQA dataset DIQA-5000.<n>The DIQA-5000 dataset comprises 5,000 document images, generated by applying multiple document enhancement techniques to 500 real-world images with diverse distortions.<n>Each enhanced image was rated by 15 subjects across three rating dimensions: overall quality, sharpness, and color fidelity.<n>We propose a specialized no-reference DIQA model that exploits document layout features to maintain quality perception at reduced resolutions to lower computational cost.
  arXiv  Detail & Related papers  (2025-09-21T10:01:43Z)
• WordVIS: A Color Worth A Thousand Words [4.891597567642704]
  Multi-modal approaches have become increasingly popular for document classification.<n>These approaches are underutilized due to their requirement for a tremendous volume of training data and extensive computational power.<n>In this paper, we attempt to address these issues by embedding textual features directly into the visual space.
  arXiv  Detail & Related papers  (2024-12-13T14:12:55Z)
• LOCR: Location-Guided Transformer for Optical Character Recognition [55.195165959662795]
  We propose LOCR, a model that integrates location guiding into the transformer architecture during autoregression. We train the model on a dataset comprising over 77M text-location pairs from 125K academic document pages, including bounding boxes for words, tables and mathematical symbols. It outperforms all existing methods in our test set constructed from arXiv, as measured by edit distance, BLEU, METEOR and F-measure.
  arXiv  Detail & Related papers  (2024-03-04T15:34:12Z)
• Filter & Align: Leveraging Human Knowledge to Curate Image-Text Data [31.507451966555383]
  We present a novel algorithm that incorporates human knowledge on image-text alignment to guide filtering vast corpus of web-crawled image-text datasets. We collect a diverse image-text dataset where each image is associated with multiple captions from various sources. We train a reward model on these human-preference annotations to internalize the nuanced human understanding of image-text alignment.
  arXiv  Detail & Related papers  (2023-12-11T05:57:09Z)
• OBELICS: An Open Web-Scale Filtered Dataset of Interleaved Image-Text Documents [122.55393759474181]
  We introduce OBELICS, an open web-scale filtered dataset of interleaved image-text documents. We describe the dataset creation process, present comprehensive filtering rules, and provide an analysis of the dataset's content. We train vision and language models of 9 and 80 billion parameters named IDEFICS, and obtain competitive performance on different multimodal benchmarks.
  arXiv  Detail & Related papers  (2023-06-21T14:01:01Z)
• DWT-CompCNN: Deep Image Classification Network for High Throughput JPEG 2000 Compressed Documents [0.9405458160620535]
  DWT CompCNN is proposed for classification of documents that are compressed using High Throughput JPEG 2000 (HTJ2K) algorithm. The proposed model is time and space efficient, and also achieves a better classification accuracy in compressed domain.
  arXiv  Detail & Related papers  (2023-06-02T08:33:58Z)
• Learning Customized Visual Models with Retrieval-Augmented Knowledge [104.05456849611895]
  We propose REACT, a framework to acquire the relevant web knowledge to build customized visual models for target domains. We retrieve the most relevant image-text pairs from the web-scale database as external knowledge, and propose to customize the model by only training new modualized blocks while freezing all the original weights. The effectiveness of REACT is demonstrated via extensive experiments on classification, retrieval, detection and segmentation tasks, including zero, few, and full-shot settings.
  arXiv  Detail & Related papers  (2023-01-17T18:59:06Z)
• Improving Zero-shot Generalization and Robustness of Multi-modal Models [70.14692320804178]
  Multi-modal image-text models such as CLIP and LiT have demonstrated impressive performance on image classification benchmarks. We investigate the reasons for this performance gap and find that many of the failure cases are caused by ambiguity in the text prompts. We propose a simple and efficient way to improve accuracy on such uncertain images by making use of the WordNet hierarchy.
  arXiv  Detail & Related papers  (2022-12-04T07:26:24Z)
• ContextCLIP: Contextual Alignment of Image-Text pairs on CLIP visual representations [4.588028371034406]
  We propose ContextCLIP, a contextual and contrastive learning framework for the contextual alignment of image-text pairs. Our framework was observed to improve the image-text alignment by aligning text and image representations contextually in the joint embedding space. ContextCLIP showed good qualitative performance for text-to-image retrieval tasks and enhanced classification accuracy.
  arXiv  Detail & Related papers  (2022-11-14T05:17:51Z)
• PyramidCLIP: Hierarchical Feature Alignment for Vision-language Model Pretraining [68.84339672878066]
  We introduce PyramidCLIP, which constructs an input pyramid with different semantic levels, and aligns visual elements and linguistic elements in the form of hierarchy. Experiments on three downstream tasks, including zero-shot image classification, zero-shot image-text retrieval and image object detection, verify the effectiveness of the proposed PyramidCLIP.
  arXiv  Detail & Related papers  (2022-04-29T13:38:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.