Digitizing Historical Balance Sheet Data: A Practitioner's Guide

• URL: http://arxiv.org/abs/2204.00052v1
• Date: Thu, 31 Mar 2022 19:18:38 GMT
• Title: Digitizing Historical Balance Sheet Data: A Practitioner's Guide
• Authors: Sergio Correia, Stephan Luck
• Abstract summary: This paper discusses how to successfully digitize large-scale historical micro-data by augmenting optical character recognition (OCR) engines with pre- and post-processing methods. We apply them against two large balance sheet datasets and introduce "quipucamayoc", a Python package containing these methods in a unified framework.
• Score: 0.30458514384586394
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper discusses how to successfully digitize large-scale historical micro-data by augmenting optical character recognition (OCR) engines with pre- and post-processing methods. Although OCR software has improved dramatically in recent years due to improvements in machine learning, off-the-shelf OCR applications still present high error rates which limits their applications for accurate extraction of structured information. Complementing OCR with additional methods can however dramatically increase its success rate, making it a powerful and cost-efficient tool for economic historians. This paper showcases these methods and explains why they are useful. We apply them against two large balance sheet datasets and introduce "quipucamayoc", a Python package containing these methods in a unified framework.

Related papers

• Reference-Based Post-OCR Processing with LLM for Diacritic Languages [0.0]
  We propose a method utilizing available content-focused ebooks as a reference base to correct imperfect OCR-generated text. This technique generates high-precision pseudo-page-to-page labels for diacritic languages. The pipeline eliminates various types of noise from aged documents and addresses issues such as missing characters, words, and disordered sequences.
  arXiv  Detail & Related papers  (2024-10-17T08:05:02Z)
• Enhancing Legal Case Retrieval via Scaling High-quality Synthetic Query-Candidate Pairs [67.54302101989542]
  Legal case retrieval aims to provide similar cases as references for a given fact description. Existing works mainly focus on case-to-case retrieval using lengthy queries. Data scale is insufficient to satisfy the training requirements of existing data-hungry neural models.
  arXiv  Detail & Related papers  (2024-10-09T06:26:39Z)
• EASRec: Elastic Architecture Search for Efficient Long-term Sequential Recommender Systems [82.76483989905961]
  Current Sequential Recommender Systems (SRSs) suffer from computational and resource inefficiencies. We develop the Elastic Architecture Search for Efficient Long-term Sequential Recommender Systems (EASRec) EASRec introduces data-aware gates that leverage historical information from input data batch to improve the performance of the recommendation network.
  arXiv  Detail & Related papers  (2024-02-01T07:22:52Z)
• Noise-Robust Dense Retrieval via Contrastive Alignment Post Training [89.29256833403167]
  Contrastive Alignment POst Training (CAPOT) is a highly efficient finetuning method that improves model robustness without requiring index regeneration. CAPOT enables robust retrieval by freezing the document encoder while the query encoder learns to align noisy queries with their unaltered root. We evaluate CAPOT noisy variants of MSMARCO, Natural Questions, and Trivia QA passage retrieval, finding CAPOT has a similar impact as data augmentation with none of its overhead.
  arXiv  Detail & Related papers  (2023-04-06T22:16:53Z)
• User-Centric Evaluation of OCR Systems for Kwak'wala [92.73847703011353]
  We show that utilizing OCR reduces the time spent in the manual transcription of culturally valuable documents by over 50%. Our results demonstrate the potential benefits that OCR tools can have on downstream language documentation and revitalization efforts.
  arXiv  Detail & Related papers  (2023-02-26T21:41:15Z)
• Donut: Document Understanding Transformer without OCR [17.397447819420695]
  We propose a novel VDU model that is end-to-end trainable without underpinning OCR framework. Our approach achieves state-of-the-art performance on various document understanding tasks in public benchmark datasets and private industrial service datasets.
  arXiv  Detail & Related papers  (2021-11-30T18:55:19Z)
• Lexically Aware Semi-Supervised Learning for OCR Post-Correction [90.54336622024299]
  Much of the existing linguistic data in many languages of the world is locked away in non-digitized books and documents. Previous work has demonstrated the utility of neural post-correction methods on recognition of less-well-resourced languages. We present a semi-supervised learning method that makes it possible to utilize raw images to improve performance.
  arXiv  Detail & Related papers  (2021-11-04T04:39:02Z)
• PP-OCRv2: Bag of Tricks for Ultra Lightweight OCR System [9.376162696601238]
  We introduce bag of tricks to train a better text detector and a better text recognizer. Experiments on real data show that the precision of PP-OCRv2 is 7% higher than PP-OCR under the same inference cost.
  arXiv  Detail & Related papers  (2021-09-07T15:24:40Z)
• Lights, Camera, Action! A Framework to Improve NLP Accuracy over OCR documents [2.6201102730518606]
  We demonstrate an effective framework for mitigating OCR errors for any downstream NLP task. We first address the data scarcity problem for model training by constructing a document synthesis pipeline. For the benefit of the community, we have made the document synthesis pipeline available as an open-source project.
  arXiv  Detail & Related papers  (2021-08-06T00:32:54Z)
• Making Online Sketching Hashing Even Faster [63.16042585506435]
  We present a FasteR Online Sketching Hashing (FROSH) algorithm to sketch the data in a more compact form via an independent transformation. We provide theoretical justification to guarantee that our proposed FROSH consumes less time and achieves a comparable sketching precision. We also extend FROSH to its distributed implementation, namely DFROSH, to further reduce the training time cost of FROSH.
  arXiv  Detail & Related papers  (2020-10-10T08:50:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.