Mixture-of-experts for handwriting trajectory reconstruction from IMU sensors

• URL: http://arxiv.org/abs/2607.26708v1
• Date: Wed, 29 Jul 2026 09:53:26 GMT
• Title: Mixture-of-experts for handwriting trajectory reconstruction from IMU sensors
• Abstract summary: We focus on a digital pen equipped with sensors where we aim at reconstructing the online handwriting trajectory.<n>This pen enables writing on any surface and preserving the digital trace of handwriting.
• Score: 2.3759453477621864
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The use of digital pens for online handwriting trajectory reconstruction is a prevalent method for human-computer interaction. In this study, we focus on a digital pen equipped with sensors where we aim at reconstructing the online handwriting trajectory. This pen enables writing on any surface and preserving the digital trace of handwriting. This type of pen could be used as an aid to learning to write in classroom. In this paper, we propose a new approach learning to finely reconstruct the touching trajectories while precisely analyzing the hovering part in order to position the next touching trace correctly. This relies on a Mixture-Of-Experts (MOE) approach. The first expert is dedicated for the pencil touch, and is named touching expert model. The second one is dedicated for the hovering pen trajectory, and is named hovering expert model. We improve on the learning of each of these experts based on additional context or specific examples. In addition we introduce a novel public benchmark dataset, to enable future research and comparisons in the field of handwriting reconstruction. The results demonstrates a significant enhancement compared to its primary competitors.

Related papers

• Write on Paper and Get the Online Digital Trace:\ ewline A New Era for Handwriting [0.04497001527881303]
  We propose an innovative solution that combines a digital pen, advanced artificial intelligence algorithms, and adaptive AI techniques to reconstruct the digital trace of handwriting.<n>This work aims to advance the state-of-the-art in automated trace reconstruction of handwriting, enabling a seamless connection between traditional handwriting on paper and capturing the trace digitally.
  arXiv  Detail & Related papers  (2026-09-11T10:53:46Z)
• Domain adaptation for handwriting trajectory reconstruction from IMU sensors [2.3759453477621864]
  This study focuses on a digital pen equipped with kinematic sensors, allowing users to write on any surface while simultaneously preserving a digital trajectory of handwriting.<n>A major issue is based on the difference in captured signals between adults and children.<n>For similar handwriting trace, we have large differences in sensor signals due to differences in speed and confidence in the handwriting gesture of children.
  arXiv  Detail & Related papers  (2026-07-29T10:29:08Z)
• Online Handwriting Trajectory Reconstruction from Kinematic Sensors using Temporal Convolutional Network [3.3424188506979147]
  We focus on a digital pen equipped with sensors from which one wants to reconstruct the Online Handwriting trajectory.<n>We introduce a novel processing pipeline that maps the sensor signals of the pen to the corresponding OH trajectory.<n>We introduce a dedicated neural network architecture, inspired by a Temporal Convolutional Network, to reconstruct the online trajectory from the pen sensor signals.
  arXiv  Detail & Related papers  (2026-07-29T10:23:18Z)
• Detecting Pen-In-Air States from Video: A Proof-of-Concept Toward Complementary Handwriting Analysis [1.8744230606349503]
  Tablet-based sensing restricts analysis of Pen-Up behavior to a short proximity range above the writing surface.<n>We propose a hybrid pipeline combining pen-tip tracking using a YOLO-based detector with kinematic feature extraction and machine learning classification.<n>Results support the feasibility of video-based Pen-Up detection as a low-cost and non-intrusive complement to digitizing tablets, and provide a foundation for future large-scale studies.
  arXiv  Detail & Related papers  (2026-06-01T14:50:27Z)
• Online Writer Retrieval with Chinese Handwritten Phrases: A Synergistic Temporal-Frequency Representation Learning Approach [53.189911918976655]
  We propose DOLPHIN, a novel retrieval model designed to enhance handwriting representations through synergistic temporal-frequency analysis.<n>We introduce OLIWER, a large-scale online writer retrieval dataset encompassing over 670,000 Chinese handwritten phrases from 1,731 individuals.<n>Our findings emphasize the significance of point sampling frequency and pressure features in improving handwriting representation quality.
  arXiv  Detail & Related papers  (2024-12-16T11:19:22Z)
• Lessons from Learning to Spin "Pens" [51.9182692233916]
  In this work, we push the boundaries of learning-based in-hand manipulation systems by demonstrating the capability to spin pen-like objects. We first use reinforcement learning to train an oracle policy with privileged information and generate a high-fidelity trajectory dataset in simulation. We then fine-tune the sensorimotor policy using these real-world trajectories to adapt it to the real world dynamics.
  arXiv  Detail & Related papers  (2024-07-26T17:56:01Z)
• InkSight: Offline-to-Online Handwriting Conversion by Teaching Vision-Language Models to Read and Write [7.4539464693425925]
  InkSight aims to empower physical note-takers to effortlessly convert their work (offline handwriting) to digital ink (online handwriting)<n>Our approach combines reading and writing priors, allowing training a model in the absence of large amounts of paired samples.<n>Our human evaluation reveals that 87% of the samples produced by our model on the challenging HierText dataset are considered as a valid tracing of the input image.
  arXiv  Detail & Related papers  (2024-02-08T16:41:41Z)
• Sampling and Ranking for Digital Ink Generation on a tight computational budget [69.15275423815461]
  We study ways to maximize the quality of the output of a trained digital ink generative model. We use and compare the effect of multiple sampling and ranking techniques, in the first ablation study of its kind in the digital ink domain.
  arXiv  Detail & Related papers  (2023-06-02T09:55:15Z)
• Tactile-Filter: Interactive Tactile Perception for Part Mating [54.46221808805662]
  Humans rely on touch and tactile sensing for a lot of dexterous manipulation tasks. vision-based tactile sensors are being widely used for various robotic perception and control tasks. We present a method for interactive perception using vision-based tactile sensors for a part mating task.
  arXiv  Detail & Related papers  (2023-03-10T16:27:37Z)
• Letter-level Online Writer Identification [86.13203975836556]
  We focus on a novel problem, letter-level online writer-id, which requires only a few trajectories of written letters as identification cues. A main challenge is that a person often writes a letter in different styles from time to time. We refer to this problem as the variance of online writing styles (Var-O-Styles)
  arXiv  Detail & Related papers  (2021-12-06T07:21:53Z)
• Digitizing Handwriting with a Sensor Pen: A Writer-Independent Recognizer [0.2580765958706854]
  This paper presents a writer-independent system that recognizes characters written on plain paper with the use of a sensor-equipped pen. The pen provides linear acceleration, angular velocity, magnetic field, and force applied by the user, and acts as a digitizer that transforms the analogue signals of the sensors into time data while writing on regular paper. We present the results of a convolutional neural network model for letter classification and show that this approach is practical and achieves promising results for writer-independent character recognition.
  arXiv  Detail & Related papers  (2021-07-08T09:25:59Z)
• SmartPatch: Improving Handwritten Word Imitation with Patch Discriminators [67.54204685189255]
  We propose SmartPatch, a new technique increasing the performance of current state-of-the-art methods. We combine the well-known patch loss with information gathered from the parallel trained handwritten text recognition system. This leads to a more enhanced local discriminator and results in more realistic and higher-quality generated handwritten words.
  arXiv  Detail & Related papers  (2021-05-21T18:34:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.