Developing a Mixed-Methods Pipeline for Community-Oriented Digitization of Kwak'wala Legacy Texts

• URL: http://arxiv.org/abs/2506.01775v1
• Date: Mon, 02 Jun 2025 15:20:09 GMT
• Title: Developing a Mixed-Methods Pipeline for Community-Oriented Digitization of Kwak'wala Legacy Texts
• Authors: Milind Agarwal, Daisy Rosenblum, Antonios Anastasopoulos,
• Abstract summary: Kwak'wala is an Indigenous language spoken in British Columbia, Canada.<n>Over 11 volumes of the earliest texts created during the collaboration between Franz Boas and George Hunt have been scanned but remain unreadable by machines.<n>We propose using a mix of off-the-shelf OCR methods, language identification, and masking to effectively isolate Kwak'wala text.
• Score: 21.21531481916695
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Kwak'wala is an Indigenous language spoken in British Columbia, with a rich legacy of published documentation spanning more than a century, and an active community of speakers, teachers, and learners engaged in language revitalization. Over 11 volumes of the earliest texts created during the collaboration between Franz Boas and George Hunt have been scanned but remain unreadable by machines. Complete digitization through optical character recognition has the potential to facilitate transliteration into modern orthographies and the creation of other language technologies. In this paper, we apply the latest OCR techniques to a series of Kwak'wala texts only accessible as images, and discuss the challenges and unique adaptations necessary to make such technologies work for these real-world texts. Building on previous methods, we propose using a mix of off-the-shelf OCR methods, language identification, and masking to effectively isolate Kwak'wala text, along with post-correction models, to produce a final high-quality transcription.

Related papers

• ParsiPy: NLP Toolkit for Historical Persian Texts in Python [1.637832760977605]
  This work introduces ParsiPy, an NLP toolkit to handle phonetic transcriptions and analyze ancient texts.<n>ParsiPy offers modules for tokenization, lemmatization, part-of-speech tagging, phoneme-to-transliteration conversion, and word embedding.
  arXiv  Detail & Related papers  (2025-03-22T16:21:29Z)
• Puzzle Pieces Picker: Deciphering Ancient Chinese Characters with Radical Reconstruction [73.26364649572237]
  Oracle Bone Inscriptions is one of the oldest existing forms of writing in the world. A large number of Oracle Bone Inscriptions (OBI) remain undeciphered, making it one of the global challenges in paleography today. This paper introduces a novel approach, namely Puzzle Pieces Picker (P$3$), to decipher these enigmatic characters through radical reconstruction.
  arXiv  Detail & Related papers  (2024-06-05T07:34:39Z)
• Deciphering Oracle Bone Language with Diffusion Models [70.69739681961558]
  Oracle Bone Script (OBS) originated from China's Shang Dynasty approximately 3,000 years ago.<n>This paper introduces a novel approach by adopting image generation techniques, specifically through the development of Oracle Bone Script Decipher (OBSD)<n>OBSD generates vital clues for decipherment, charting a new course for AI-assisted analysis of ancient languages.
  arXiv  Detail & Related papers  (2024-06-02T09:42:23Z)
• Optical Text Recognition in Nepali and Bengali: A Transformer-based Approach [0.0]
  This paper discusses text recognition for two scripts: Bengali and Nepali. There are about 300 and 40 million Bengali and Nepali speakers respectively. The results signify that the suggested technique corresponds with current approaches.
  arXiv  Detail & Related papers  (2024-04-03T00:21:14Z)
• PHD: Pixel-Based Language Modeling of Historical Documents [55.75201940642297]
  We propose a novel method for generating synthetic scans to resemble real historical documents. We pre-train our model, PHD, on a combination of synthetic scans and real historical newspapers from the 1700-1900 period. We successfully apply our model to a historical QA task, highlighting its usefulness in this domain.
  arXiv  Detail & Related papers  (2023-10-22T08:45:48Z)
• Lexically Aware Semi-Supervised Learning for OCR Post-Correction [90.54336622024299]
  Much of the existing linguistic data in many languages of the world is locked away in non-digitized books and documents. Previous work has demonstrated the utility of neural post-correction methods on recognition of less-well-resourced languages. We present a semi-supervised learning method that makes it possible to utilize raw images to improve performance.
  arXiv  Detail & Related papers  (2021-11-04T04:39:02Z)
• One-shot Compositional Data Generation for Low Resource Handwritten Text Recognition [10.473427493876422]
  Low resource Handwritten Text Recognition is a hard problem due to the scarce annotated data and the very limited linguistic information. In this paper we address this problem through a data generation technique based on Bayesian Program Learning. Contrary to traditional generation approaches, which require a huge amount of annotated images, our method is able to generate human-like handwriting using only one sample of each symbol from the desired alphabet.
  arXiv  Detail & Related papers  (2021-05-11T18:53:01Z)
• Deep Learning for Text Style Transfer: A Survey [71.8870854396927]
  Text style transfer is an important task in natural language generation, which aims to control certain attributes in the generated text. We present a systematic survey of the research on neural text style transfer, spanning over 100 representative articles since the first neural text style transfer work in 2017. We discuss the task formulation, existing datasets and subtasks, evaluation, as well as the rich methodologies in the presence of parallel and non-parallel data.
  arXiv  Detail & Related papers  (2020-11-01T04:04:43Z)
• Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer [64.22926988297685]
  Transfer learning, where a model is first pre-trained on a data-rich task before being fine-tuned on a downstream task, has emerged as a powerful technique in natural language processing (NLP) In this paper, we explore the landscape of introducing transfer learning techniques for NLP by a unified framework that converts all text-based language problems into a text-to-text format.
  arXiv  Detail & Related papers  (2019-10-23T17:37:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.