Robustness of Structured Data Extraction from Perspectively Distorted Documents

• URL: http://arxiv.org/abs/2511.17607v1
• Date: Tue, 18 Nov 2025 07:54:21 GMT
• Title: Robustness of Structured Data Extraction from Perspectively Distorted Documents
• Abstract summary: Real-world document images are usually not only in-plane rotated but also perspectively distorted.<n>This study investigates the impacts of such perturbations on the data extraction accuracy for the state-of-the-art model, Gemini-1.5-pro.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Optical Character Recognition (OCR) for data extraction from documents is essential to intelligent informatics, such as digitizing medical records and recognizing road signs. Multi-modal Large Language Models (LLMs) can solve this task and have shown remarkable performance. Recently, it has been noticed that the accuracy of data extraction by multi-modal LLMs can be affected when in-plane rotations are present in the documents. However, real-world document images are usually not only in-plane rotated but also perspectively distorted. This study investigates the impacts of such perturbations on the data extraction accuracy for the state-of-the-art model, Gemini-1.5-pro. Because perspective distortions have a high degree of freedom, designing experiments in the same manner as single-parametric rotations is difficult. We observed typical distortions of document images and showed that most of them approximately follow an isosceles-trapezoidal transformation, which allows us to evaluate distortions with a small number of parameters. We were able to reduce the number of independent parameters from eight to two, i.e. rotation angle and distortion ratio. Then, specific entities were extracted from synthetically generated sample documents with varying these parameters. As the performance of LLMs, we evaluated not only a character-recognition accuracy but also a structure-recognition accuracy. Whereas the former represents the classical indicators for optical character recognition, the latter is related to the correctness of reading order. In particular, the structure-recognition accuracy was found to be significantly degraded by document distortion. In addition, we found that this accuracy can be improved by a simple rotational correction. This insight will contribute to the practical use of multi-modal LLMs for OCR tasks.

Related papers

• A model of errors in transformers [14.482123927397135]
  We study the error rate of LLMs on tasks that require a deterministic output, and repetitive processing of tokens drawn from a small set of alternatives.<n>We argue that incorrect predictions arise when small errors in the attention mechanism accumulate to cross a threshold.<n>We show how to construct prompts to reduce the error rate.
  arXiv  Detail & Related papers  (2026-01-20T17:27:03Z)
• PaperAudit-Bench: Benchmarking Error Detection in Research Papers for Critical Automated Peer Review [54.141490756509306]
  We introduce PaperAudit-Bench, which consists of two components: PaperAudit-Dataset, an error dataset, and PaperAudit-Review, an automated review framework.<n>Experiments on PaperAudit-Bench reveal large variability in error detectability across models and detection depths.<n>We show that the dataset supports training lightweight LLM detectors via SFT and RL, enabling effective error detection at reduced computational cost.
  arXiv  Detail & Related papers  (2026-01-07T04:26:12Z)
• Cascaded Robust Rectification for Arbitrary Document Images [45.30113042855903]
  Document rectification in real-world scenarios poses significant challenges due to extreme variations in camera perspectives and physical distortions.<n>We introduce a novel multi-stage framework that progressively reverses distinct distortion types in a coarse-to-fine manner.<n>Our framework first performs a global affine transformation to correct perspective distortions arising from the camera's viewpoint, then rectifies geometric deformations resulting from physical paper curling and folding, and finally employs a content-aware iterative process to eliminate fine-grained content distortions.
  arXiv  Detail & Related papers  (2025-11-28T12:56:16Z)
• Seeing Straight: Document Orientation Detection for Efficient OCR [2.7873355152549344]
  OCR-Rotation-Bench (ORB) is a new benchmark for evaluating OCR to image rotations.<n>We present a fast, robust and lightweight rotation classification pipeline built on the vision encoder of Phi-3.5-Vision model.<n>Our method achieves near-perfect 96% and 92% accuracy on identifying the rotations respectively on both the datasets.
  arXiv  Detail & Related papers  (2025-11-06T08:04:57Z)
• Beyond Frequency: Scoring-Driven Debiasing for Object Detection via Blueprint-Prompted Image Synthesis [97.37770785712475]
  We present a generation-based debiasing framework for object detection.<n>Our method significantly narrows the performance gap for underrepresented object groups.
  arXiv  Detail & Related papers  (2025-10-21T02:19:12Z)
• Registration is a Powerful Rotation-Invariance Learner for 3D Anomaly Detection [64.0168648353038]
  3D anomaly detection in point-cloud data is critical for industrial quality control, aiming to identify structural defects with high reliability.<n>Current memory bank-based methods often suffer from inconsistent feature transformations and limited discriminative capacity.<n>We propose a registration-induced, rotation-invariant feature extraction framework that integrates the objectives of point-cloud registration and memory-based anomaly detection.
  arXiv  Detail & Related papers  (2025-10-19T14:56:38Z)
• Boosting Semi-Supervised Scene Text Recognition via Viewing and Summarizing [71.29488677105127]
  Existing scene text recognition (STR) methods struggle to recognize challenging texts, especially for artistic and severely distorted characters. We propose a contrastive learning-based STR framework by leveraging synthetic and real unlabeled data without any human cost. Our method achieves SOTA performance (94.7% and 70.9% average accuracy on common benchmarks and Union14M-Benchmark.
  arXiv  Detail & Related papers  (2024-11-23T15:24:47Z)
• Mind the Gap: Analyzing Lacunae with Transformer-Based Transcription [2.486161976966064]
  We employ transformer-based optical character recognition (OCR) models trained on synthetic data containing lacunae. We demonstrate their effectiveness in detecting and restoring lacunae, achieving a success rate of 65%.
  arXiv  Detail & Related papers  (2024-06-28T22:52:39Z)
• Robustness of Structured Data Extraction from In-plane Rotated Documents using Multi-Modal Large Language Models (LLM) [0.0]
  This study investigates the impact of document skew on the data extraction accuracy of three state-of-the-art multi-modal models. We identify the safe in-plane rotation angles (SIPRA) for each model and investigate the effects of skew on model hallucinations.
  arXiv  Detail & Related papers  (2024-06-13T08:55:01Z)
• Produce Once, Utilize Twice for Anomaly Detection [6.501323305130114]
  We derive POUTA, which improves both the accuracy and efficiency by reusing the discriminant information potential in the reconstructive network. POUTA achieves better performance than the state-of-the-art few-shot anomaly detection methods without any special design.
  arXiv  Detail & Related papers  (2023-12-20T10:49:49Z)
• End-to-End Page-Level Assessment of Handwritten Text Recognition [69.55992406968495]
  HTR systems increasingly face the end-to-end page-level transcription of a document. Standard metrics do not take into account the inconsistencies that might appear. We propose a two-fold evaluation, where the transcription accuracy and the RO goodness are considered separately.
  arXiv  Detail & Related papers  (2023-01-14T15:43:07Z)
• Lights, Camera, Action! A Framework to Improve NLP Accuracy over OCR documents [2.6201102730518606]
  We demonstrate an effective framework for mitigating OCR errors for any downstream NLP task. We first address the data scarcity problem for model training by constructing a document synthesis pipeline. For the benefit of the community, we have made the document synthesis pipeline available as an open-source project.
  arXiv  Detail & Related papers  (2021-08-06T00:32:54Z)
• Learning High-Precision Bounding Box for Rotated Object Detection via Kullback-Leibler Divergence [100.6913091147422]
  Existing rotated object detectors are mostly inherited from the horizontal detection paradigm. In this paper, we are motivated to change the design of rotation regression loss from induction paradigm to deduction methodology.
  arXiv  Detail & Related papers  (2021-06-03T14:29:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.