WordVIS: A Color Worth A Thousand Words

• URL: http://arxiv.org/abs/2412.10155v1
• Date: Fri, 13 Dec 2024 14:12:55 GMT
• Title: WordVIS: A Color Worth A Thousand Words
• Abstract summary: Multi-modal approaches have become increasingly popular for document classification.<n>These approaches are underutilized due to their requirement for a tremendous volume of training data and extensive computational power.<n>In this paper, we attempt to address these issues by embedding textual features directly into the visual space.
• Score: 4.891597567642704
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Document classification is considered a critical element in automated document processing systems. In recent years multi-modal approaches have become increasingly popular for document classification. Despite their improvements, these approaches are underutilized in the industry due to their requirement for a tremendous volume of training data and extensive computational power. In this paper, we attempt to address these issues by embedding textual features directly into the visual space, allowing lightweight image-based classifiers to achieve state-of-the-art results using small-scale datasets in document classification. To evaluate the efficacy of the visual features generated from our approach on limited data, we tested on the standard dataset Tobacco-3482. Our experiments show a tremendous improvement in image-based classifiers, achieving an improvement of 4.64% using ResNet50 with no document pre-training. It also sets a new record for the best accuracy of the Tobacco-3482 dataset with a score of 91.14% using the image-based DocXClassifier with no document pre-training. The simplicity of the approach, its resource requirements, and subsequent results provide a good prospect for its use in industrial use cases.

Related papers

• Bounding Box Label Propagation for Re-Annotation of Document Layout Analysis Datasets [49.345169562968415]
  Bounding Box Label Propagation (BBLP) is a pseudo-labelling framework for object detection.<n>It produces high-quality class annotations of bounding boxes.<n>It achieves a mAP of 54.0%, corresponding to 81.6% of fully supervised performance, while using only 10% labelled data.
  arXiv  Detail & Related papers  (2026-06-16T08:04:27Z)
• Page image classifier fine-tuned on century-spanning archives of scanned documents for further content-specific processing [0.0]
  Digitization projects in the humanities produce vast, heterogeneous archives of historical documents.<n>This work addresses the need for an automated system to classify scanned page images based on visual content type.<n>An image classification system was developed and evaluated on a dataset of over 48,000 annotated historical page images.
  arXiv  Detail & Related papers  (2026-05-25T08:36:36Z)
• DocReward: A Document Reward Model for Structuring and Stylizing [107.03974018371058]
  DocReward is a document reward model that evaluates documents based on their structure and style.<n>It is trained using the Bradley-Terry loss to score documents, penalizing predictions that contradict the annotated ranking.<n>It achieves a significantly higher win rate of 60.8%, compared to GPT-5's 37.7% win rate.
  arXiv  Detail & Related papers  (2025-10-13T13:36:32Z)
• Learning Customized Visual Models with Retrieval-Augmented Knowledge [104.05456849611895]
  We propose REACT, a framework to acquire the relevant web knowledge to build customized visual models for target domains. We retrieve the most relevant image-text pairs from the web-scale database as external knowledge, and propose to customize the model by only training new modualized blocks while freezing all the original weights. The effectiveness of REACT is demonstrated via extensive experiments on classification, retrieval, detection and segmentation tasks, including zero, few, and full-shot settings.
  arXiv  Detail & Related papers  (2023-01-17T18:59:06Z)
• Revisiting Classifier: Transferring Vision-Language Models for Video Recognition [102.93524173258487]
  Transferring knowledge from task-agnostic pre-trained deep models for downstream tasks is an important topic in computer vision research. In this study, we focus on transferring knowledge for video classification tasks. We utilize the well-pretrained language model to generate good semantic target for efficient transferring learning.
  arXiv  Detail & Related papers  (2022-07-04T10:00:47Z)
• Annotation Error Detection: Analyzing the Past and Present for a More Coherent Future [63.99570204416711]
  We reimplement 18 methods for detecting potential annotation errors and evaluate them on 9 English datasets. We define a uniform evaluation setup including a new formalization of the annotation error detection task. We release our datasets and implementations in an easy-to-use and open source software package.
  arXiv  Detail & Related papers  (2022-06-05T22:31:45Z)
• Budget-aware Few-shot Learning via Graph Convolutional Network [56.41899553037247]
  This paper tackles the problem of few-shot learning, which aims to learn new visual concepts from a few examples. A common problem setting in few-shot classification assumes random sampling strategy in acquiring data labels. We introduce a new budget-aware few-shot learning problem that aims to learn novel object categories.
  arXiv  Detail & Related papers  (2022-01-07T02:46:35Z)
• Domain Agnostic Few-Shot Learning For Document Intelligence [4.243926243206826]
  Few-shot learning aims to generalize to novel classes with only a few samples with class labels. In this work, we address the problem of few-shot document image classification under domain shift.
  arXiv  Detail & Related papers  (2021-10-29T03:19:31Z)
• Efficient Document Image Classification Using Region-Based Graph Neural Network [4.147346416230273]
  Document image classification remains a popular research area because it can be commercialized in many enterprise applications across different industries. Recent advancements in large pre-trained computer vision and language models and graph neural networks has lent document image classification many tools. We propose an efficient document image classification framework that uses graph convolution neural networks.
  arXiv  Detail & Related papers  (2021-06-25T17:57:04Z)
• Towards Good Practices for Efficiently Annotating Large-Scale Image Classification Datasets [90.61266099147053]
  We investigate efficient annotation strategies for collecting multi-class classification labels for a large collection of images. We propose modifications and best practices aimed at minimizing human labeling effort. Simulated experiments on a 125k image subset of the ImageNet100 show that it can be annotated to 80% top-1 accuracy with 0.35 annotations per image on average.
  arXiv  Detail & Related papers  (2021-04-26T16:29:32Z)
• Robust Layout-aware IE for Visually Rich Documents with Pre-trained Language Models [23.42593796135709]
  We study the problem of information extraction from visually rich documents (VRDs) We present a model that combines the power of large pre-trained language models and graph neural networks to efficiently encode both textual and visual information in business documents.
  arXiv  Detail & Related papers  (2020-05-22T06:04:50Z)
• Self-Supervised Representation Learning on Document Images [8.927538538637783]
  We show that patch-based pre-training performs poorly on document images because of their different structural properties and poor intra-sample semantic information. We propose two context-aware alternatives to improve performance on the Tobacco-3482 image classification task.
  arXiv  Detail & Related papers  (2020-04-18T10:14:06Z)
• Selecting Relevant Features from a Multi-domain Representation for Few-shot Classification [91.67977602992657]
  We propose a new strategy based on feature selection, which is both simpler and more effective than previous feature adaptation approaches. We show that a simple non-parametric classifier built on top of such features produces high accuracy and generalizes to domains never seen during training.
  arXiv  Detail & Related papers  (2020-03-20T15:44:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.