Enhancing IMU-Based Online Handwriting Recognition via Contrastive Learning with Zero Inference Overhead

• URL: http://arxiv.org/abs/2602.07049v1
• Date: Wed, 04 Feb 2026 13:44:54 GMT
• Title: Enhancing IMU-Based Online Handwriting Recognition via Contrastive Learning with Zero Inference Overhead
• Authors: Jindong Li, Dario Zanca, Vincent Christlein, Tim Hamann, Jens Barth, Peter Kämpf, Björn Eskofier,
• Abstract summary: We propose a training framework designed to improve feature representation and recognition accuracy without increasing inference costs.<n>ECHWR utilizes a temporary auxiliary branch that aligns sensor signals with semantic text embeddings during the training phase.<n> Evaluations on the OnHW-Words500 dataset show that ECHWR significantly outperforms state-of-the-art baselines.
• Score: 4.519836503888727
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Online handwriting recognition using inertial measurement units opens up handwriting on paper as input for digital devices. Doing it on edge hardware improves privacy and lowers latency, but entails memory constraints. To address this, we propose Error-enhanced Contrastive Handwriting Recognition (ECHWR), a training framework designed to improve feature representation and recognition accuracy without increasing inference costs. ECHWR utilizes a temporary auxiliary branch that aligns sensor signals with semantic text embeddings during the training phase. This alignment is maintained through a dual contrastive objective: an in-batch contrastive loss for general modality alignment and a novel error-based contrastive loss that distinguishes between correct signals and synthetic hard negatives. The auxiliary branch is discarded after training, which allows the deployed model to keep its original, efficient architecture. Evaluations on the OnHW-Words500 dataset show that ECHWR significantly outperforms state-of-the-art baselines, reducing character error rates by up to 7.4% on the writer-independent split and 10.4% on the writer-dependent split. Finally, although our ablation studies indicate that solving specific challenges require specific architectural and objective configurations, error-based contrastive loss shows its effectiveness for handling unseen writing styles.

Related papers

• MetaWriter: Personalized Handwritten Text Recognition Using Meta-Learned Prompt Tuning [6.274266343486906]
  Traditional handwritten text recognition methods lack writer-specific personalization at test time.<n>We propose an efficient framework that formulates personalization as prompt tuning.<n>We validate our approach on the RIMES and IAM Handwriting Database benchmarks.
  arXiv  Detail & Related papers  (2025-05-26T20:26:16Z)
• Preserving Privacy Without Compromising Accuracy: Machine Unlearning for Handwritten Text Recognition [12.022436561452766]
  Regulations such as the right to be forgotten'' require models to remove sensitive traces without full retraining.<n>We propose a two-stage unlearning framework for multihead transformer HTR models.<n>We also present Writer-ID Confusion (WIC), a method that forces the forget set to follow a uniform distribution over writer identities.
  arXiv  Detail & Related papers  (2025-04-11T15:21:12Z)
• Privacy-Preserving Biometric Verification with Handwritten Random Digit String [49.77172854374479]
  Handwriting verification has stood as a steadfast identity authentication method for decades.<n>However, this technique risks potential privacy breaches due to the inclusion of personal information in handwritten biometrics such as signatures.<n>We propose using the Random Digit String (RDS) for privacy-preserving handwriting verification.
  arXiv  Detail & Related papers  (2025-03-17T03:47:25Z)
• Robust and Efficient Writer-Independent IMU-Based Handwriting Recognition [2.022040536958726]
  Online handwriting recognition (HWR) using data from inertial measurement units (IMUs) remains challenging due to variations in writing styles.<n>Previous approaches often struggle with handwriting from unseen writers, making writer-independent (WI) recognition a crucial yet difficult problem.<n>This paper presents an HWR model designed to improve WI HWR on IMU data, using a CNN encoder and a BiLSTM-based decoder.
  arXiv  Detail & Related papers  (2025-02-28T11:09:28Z)
• Online Writer Retrieval with Chinese Handwritten Phrases: A Synergistic Temporal-Frequency Representation Learning Approach [53.189911918976655]
  We propose DOLPHIN, a novel retrieval model designed to enhance handwriting representations through synergistic temporal-frequency analysis.<n>We introduce OLIWER, a large-scale online writer retrieval dataset encompassing over 670,000 Chinese handwritten phrases from 1,731 individuals.<n>Our findings emphasize the significance of point sampling frequency and pressure features in improving handwriting representation quality.
  arXiv  Detail & Related papers  (2024-12-16T11:19:22Z)
• Affinity-Graph-Guided Contractive Learning for Pretext-Free Medical Image Segmentation with Minimal Annotation [55.325956390997]
  This paper proposes an affinity-graph-guided semi-supervised contrastive learning framework (Semi-AGCL) for medical image segmentation. The framework first designs an average-patch-entropy-driven inter-patch sampling method, which can provide a robust initial feature space. With merely 10% of the complete annotation set, our model approaches the accuracy of the fully annotated baseline, manifesting a marginal deviation of only 2.52%.
  arXiv  Detail & Related papers  (2024-10-14T10:44:47Z)
• Offline Detection of Misspelled Handwritten Words by Convolving Recognition Model Features with Text Labels [0.0]
  We introduce the task of comparing a handwriting image to text. Our model's classification head is trained entirely on synthetic data created using a state-of-the-art generative adversarial network. Such massive performance gains can lead to significant productivity increases in applications utilizing human-in-the-loop automation.
  arXiv  Detail & Related papers  (2023-09-18T21:13:42Z)
• End-to-End Page-Level Assessment of Handwritten Text Recognition [69.55992406968495]
  HTR systems increasingly face the end-to-end page-level transcription of a document. Standard metrics do not take into account the inconsistencies that might appear. We propose a two-fold evaluation, where the transcription accuracy and the RO goodness are considered separately.
  arXiv  Detail & Related papers  (2023-01-14T15:43:07Z)
• SURDS: Self-Supervised Attention-guided Reconstruction and Dual Triplet Loss for Writer Independent Offline Signature Verification [16.499360910037904]
  Offline Signature Verification (OSV) is a fundamental biometric task across various forensic, commercial and legal applications. We propose a two-stage deep learning framework that leverages self-supervised representation learning as well as metric learning for writer-independent OSV. The proposed framework has been evaluated on two publicly available offline signature datasets and compared with various state-of-the-art methods.
  arXiv  Detail & Related papers  (2022-01-25T07:26:55Z)
• SmartPatch: Improving Handwritten Word Imitation with Patch Discriminators [67.54204685189255]
  We propose SmartPatch, a new technique increasing the performance of current state-of-the-art methods. We combine the well-known patch loss with information gathered from the parallel trained handwritten text recognition system. This leads to a more enhanced local discriminator and results in more realistic and higher-quality generated handwritten words.
  arXiv  Detail & Related papers  (2021-05-21T18:34:21Z)
• Label-Synchronous Speech-to-Text Alignment for ASR Using Forward and Backward Transformers [49.403414751667135]
  This paper proposes a novel label-synchronous speech-to-text alignment technique for automatic speech recognition (ASR) The proposed method re-defines the speech-to-text alignment as a label-synchronous text mapping problem. Experiments using the corpus of spontaneous Japanese (CSJ) demonstrate that the proposed method provides an accurate utterance-wise alignment.
  arXiv  Detail & Related papers  (2021-04-21T03:05:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.