Zero-Shot Synthetic-to-Real Handwritten Text Recognition via Task Analogies

• URL: http://arxiv.org/abs/2604.09713v1
• Date: Wed, 08 Apr 2026 07:33:31 GMT
• Title: Zero-Shot Synthetic-to-Real Handwritten Text Recognition via Task Analogies
• Abstract summary: Handwritten Text Recognition (HTR) models often struggle to generalize to real text.<n>We tackle the fully zero-shot synthetic-to-real generalization setting.<n>Our approach learns how model parameters change when moving from synthetic to real handwriting in one or more source languages.
• Score: 60.84720484975815
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Handwritten Text Recognition (HTR) models trained on synthetic handwriting often struggle to generalize to real text, and existing adaptation methods still require real samples from the target domain. In this work, we tackle the fully zero-shot synthetic-to-real generalization setting, where no real data from the target language is available. Our approach learns how model parameters change when moving from synthetic to real handwriting in one or more source languages and transfers this learned correction to new target languages. When using multiple sources, we rely on linguistic similarity to weigh their contrubition when combining them. Experiments across five languages and six architectures show consistent improvements over synthetic-only baselines and reveal that the transferred corrections benefit even languages unrelated to the sources.

Related papers

• How Far Do Simple Transformations Translate Across Text Embedding Models? [1.7812673697676977]
  We investigate whether simple transformations can translate representations across heterogeneous text embedding models.<n>We test the hypothesis of latent universality in a realistic text setting beyond simplified benchmarks.<n>Overall, the results show that heterogeneous embedding spaces are not universally related by simple mappings as often suggested in some literature.
  arXiv  Detail & Related papers  (2026-08-06T12:57:01Z)
• Language-Agnostic Visual Embeddings for Cross-Script Handwriting Retrieval [5.359439761925416]
  We propose a lightweight asymmetric dual-encoder framework that learns unified, style-invariant visual embeddings.<n>By jointly optimizing instance-level alignment and class-level semantic consistency, our approach anchors visual embeddings to language-agnostic semantic prototypes.<n> Experiments show that our method outperforms 28 baselines and state-of-the-art accuracy on within-language retrieval benchmarks.
  arXiv  Detail & Related papers  (2026-01-16T12:55:41Z)
• STELLAR: Scene Text Editor for Low-Resource Languages and Real-World Data [3.622341086373503]
  We propose STELLAR, a Scene Text Editor for Low-resource LAnguages and Real-world data.<n> STELLAR enables reliable multilingual editing through a language-adaptive glyph encoder and a multi-stage training strategy.<n>We also construct a new dataset, STIPLAR(Scene Text Image Pairs of Low-resource lAnguages and Real-world data), for training and evaluation.
  arXiv  Detail & Related papers  (2025-11-13T05:18:03Z)
• Bridging Synthetic and Real Worlds for Pre-training Scene Text Detectors [54.80516786370663]
  FreeReal is a real-domain-aligned pre-training paradigm that enables the complementary strengths of LSD and real data. GlyphMix embeds synthetic images as graffiti-like units onto real images. FreeReal consistently outperforms previous pre-training methods by a substantial margin across four public datasets.
  arXiv  Detail & Related papers  (2023-12-08T15:10:55Z)
• WordStylist: Styled Verbatim Handwritten Text Generation with Latent Diffusion Models [8.334487584550185]
  We present a latent diffusion-based method for styled text-to-text-content-image generation on word-level. Our proposed method is able to generate realistic word image samples from different writer styles. We show that the proposed model produces samples that are aesthetically pleasing, help boosting text recognition performance, and get similar writer retrieval score as real data.
  arXiv  Detail & Related papers  (2023-03-29T10:19:26Z)
• The Whole Truth and Nothing But the Truth: Faithful and Controllable Dialogue Response Generation with Dataflow Transduction and Constrained Decoding [65.34601470417967]
  We describe a hybrid architecture for dialogue response generation that combines the strengths of neural language modeling and rule-based generation. Our experiments show that this system outperforms both rule-based and learned approaches in human evaluations of fluency, relevance, and truthfulness.
  arXiv  Detail & Related papers  (2022-09-16T09:00:49Z)
• Linking Emergent and Natural Languages via Corpus Transfer [98.98724497178247]
  We propose a novel way to establish a link by corpus transfer between emergent languages and natural languages. Our approach showcases non-trivial transfer benefits for two different tasks -- language modeling and image captioning. We also introduce a novel metric to predict the transferability of an emergent language by translating emergent messages to natural language captions grounded on the same images.
  arXiv  Detail & Related papers  (2022-03-24T21:24:54Z)
• How much do language models copy from their training data? Evaluating linguistic novelty in text generation using RAVEN [63.79300884115027]
  Current language models can generate high-quality text. Are they simply copying text they have seen before, or have they learned generalizable linguistic abstractions? We introduce RAVEN, a suite of analyses for assessing the novelty of generated text.
  arXiv  Detail & Related papers  (2021-11-18T04:07:09Z)
• Lexically Aware Semi-Supervised Learning for OCR Post-Correction [90.54336622024299]
  Much of the existing linguistic data in many languages of the world is locked away in non-digitized books and documents. Previous work has demonstrated the utility of neural post-correction methods on recognition of less-well-resourced languages. We present a semi-supervised learning method that makes it possible to utilize raw images to improve performance.
  arXiv  Detail & Related papers  (2021-11-04T04:39:02Z)
• On The Ingredients of an Effective Zero-shot Semantic Parser [95.01623036661468]
  We analyze zero-shot learning by paraphrasing training examples of canonical utterances and programs from a grammar. We propose bridging these gaps using improved grammars, stronger paraphrasers, and efficient learning methods. Our model achieves strong performance on two semantic parsing benchmarks (Scholar, Geo) with zero labeled data.
  arXiv  Detail & Related papers  (2021-10-15T21:41:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.