Few-shot Writer Adaptation via Multimodal In-Context Learning

• URL: http://arxiv.org/abs/2603.29450v1
• Date: Tue, 31 Mar 2026 08:55:11 GMT
• Title: Few-shot Writer Adaptation via Multimodal In-Context Learning
• Abstract summary: We propose a novel context-driven HTR framework3 inspired by multimodal in-context learning.<n>We show that our approach surpasses all writer-independent HTR models without requiring any parameter updates at inference time.
• Score: 2.4314488548686
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: While state-of-the-art Handwritten Text Recognition (HTR) models perform well on standard benchmarks, they frequently struggle with writers exhibiting highly specific styles that are underrepresented in the training data. To handle unseen and atypical writers, writer adaptation techniques personalize HTR models to individual handwriting styles. Leading writer adaptation methods require either offline fine-tuning or parameter updates at inference time, both involving gradient computation and backpropagation, which increase computational costs and demand careful hyperparameter tuning. In this work, we propose a novel context-driven HTR framework3 inspired by multimodal in-context learning, enabling inference-time writer adaptation using only a few examples from the target writer without any parameter updates. We further demonstrate the impact of context length, design a compact 8M-parameter CNN-Transformer that enables few-shot in-context adaptation, and show that combining context-driven and standard OCR training strategies leads to complementary improvements. Experiments on IAM and RIMES validate our approach with Character Error Rates of 3.92% and 2.34%, respectively, surpassing all writer-independent HTR models without requiring any parameter updates at inference time.

Related papers

• MetaWriter: Personalized Handwritten Text Recognition Using Meta-Learned Prompt Tuning [6.274266343486906]
  Traditional handwritten text recognition methods lack writer-specific personalization at test time.<n>We propose an efficient framework that formulates personalization as prompt tuning.<n>We validate our approach on the RIMES and IAM Handwriting Database benchmarks.
  arXiv  Detail & Related papers  (2025-05-26T20:26:16Z)
• Personalized Text Generation with Contrastive Activation Steering [63.60368120937822]
  We propose a training-free framework that disentangles and represents personalized writing style as a vector.<n>Our framework achieves a significant 8% relative improvement in personalized generation while reducing storage requirements by 1700 times over PEFT method.
  arXiv  Detail & Related papers  (2025-03-07T08:07:15Z)
• Discourse Features Enhance Detection of Document-Level Machine-Generated Content [53.41994768824785]
  Machine-generated content poses challenges such as academic plagiarism and the spread of misinformation.<n>Existing MGC detectors often focus solely on surface-level information, overlooking implicit and structural features.<n>We introduce novel methodologies and datasets to overcome these challenges.
  arXiv  Detail & Related papers  (2024-12-17T08:47:41Z)
• Analysing Zero-Shot Readability-Controlled Sentence Simplification [54.09069745799918]
  We investigate how different types of contextual information affect a model's ability to generate sentences with the desired readability.<n>Results show that all tested models struggle to simplify sentences due to models' limitations and characteristics of the source sentences.<n>Our experiments also highlight the need for better automatic evaluation metrics tailored to RCTS.
  arXiv  Detail & Related papers  (2024-09-30T12:36:25Z)
• Efficient and Versatile Robust Fine-Tuning of Zero-shot Models [34.27380518351181]
  We introduce Robust Adapter (R-Adapter), a novel method for fine-tuning zero-shot models to downstream tasks. Our method integrates lightweight modules into the pre-trained model and employs novel self-ensemble techniques to boost OOD robustness and reduce storage expenses substantially. Our experiments demonstrate that R-Adapter achieves state-of-the-art performance across a diverse set of tasks, tuning only 13% of the parameters of the CLIP encoders.
  arXiv  Detail & Related papers  (2024-08-11T11:37:43Z)
• Repetition Improves Language Model Embeddings [86.71985212601258]
  "echo embeddings" convert autoregressive language models into strong text embedding models without changing the architecture or requiring fine-tuning.<n>Our zero-shot embeddings nearly match those obtained by bidirectionally-converted LMs that undergo additional masked-language modeling training.
  arXiv  Detail & Related papers  (2024-02-23T17:25:10Z)
• Multi-Sentence Grounding for Long-term Instructional Video [63.27905419718045]
  We aim to establish an automatic, scalable pipeline for denoising a large-scale instructional dataset. We construct a high-quality video-text dataset with multiple descriptive steps supervision, named HowToStep.
  arXiv  Detail & Related papers  (2023-12-21T17:28:09Z)
• Writer adaptation for offline text recognition: An exploration of neural network-based methods [5.285396202883411]
  An ideal handwritten text recognition (HTR) model should be adaptive to new writing styles. We show how HTR models can be made writer adaptive by using only a handful of examples from a new writer.
  arXiv  Detail & Related papers  (2023-07-11T11:35:08Z)
• MetricPrompt: Prompting Model as a Relevance Metric for Few-shot Text Classification [65.51149771074944]
  MetricPrompt eases verbalizer design difficulty by reformulating few-shot text classification task into text pair relevance estimation task. We conduct experiments on three widely used text classification datasets across four few-shot settings. Results show that MetricPrompt outperforms manual verbalizer and other automatic verbalizer design methods across all few-shot settings.
  arXiv  Detail & Related papers  (2023-06-15T06:51:35Z)
• Towards Writing Style Adaptation in Handwriting Recognition [0.0]
  We explore models with writer-dependent parameters which take the writer's identity as an additional input.<n>We propose a Writer Style Block (WSB), an adaptive instance normalization layer conditioned on learned embeddings of the partitions.<n>We show that our approach outperforms a baseline with no WSB in a writer-dependent scenario and that it is possible to estimate embeddings for new writers.
  arXiv  Detail & Related papers  (2023-02-13T12:36:17Z)
• Unsupervised Neural Stylistic Text Generation using Transfer learning and Adapters [66.17039929803933]
  We propose a novel transfer learning framework which updates only $0.3%$ of model parameters to learn style specific attributes for response generation. We learn style specific attributes from the PERSONALITY-CAPTIONS dataset.
  arXiv  Detail & Related papers  (2022-10-07T00:09:22Z)
• Stylized Knowledge-Grounded Dialogue Generation via Disentangled Template Rewriting [55.10977824136768]
  We study a new problem: Stylized Knowledge-Grounded Dialogue Generation. It presents two challenges: How to train a SKDG model where no context, knowledge, stylized response> triples are available. We propose a novel disentangled template rewriting (DTR) method which generates responses via combing disentangled style templates and content templates.
  arXiv  Detail & Related papers  (2022-04-12T08:17:21Z)
• MetaHTR: Towards Writer-Adaptive Handwritten Text Recognition [36.12001394921506]
  We propose a new approach to handwritten text recognition. We use a novel meta-learning framework which exploits additional new-writer data. Our framework can be easily implemented on the top of most state-of-the-art HTR models.
  arXiv  Detail & Related papers  (2021-04-05T12:35:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.