DocTTT: Test-Time Training for Handwritten Document Recognition Using Meta-Auxiliary Learning

• URL: http://arxiv.org/abs/2501.12898v1
• Date: Wed, 22 Jan 2025 14:18:47 GMT
• Title: DocTTT: Test-Time Training for Handwritten Document Recognition Using Meta-Auxiliary Learning
• Authors: Wenhao Gu, Li Gu, Ziqiang Wang, Ching Yee Suen, Yang Wang,
• Abstract summary: We introduce the DocTTT framework to address these challenges.<n>Key innovation of our approach is that it uses test-time training to adapt the model to each specific input during testing.<n>We propose a novel Meta-Auxiliary learning approach that combines Meta-learning and self-supervised Masked Autoencoder(MAE)
• Score: 7.036629164442979
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Despite recent significant advancements in Handwritten Document Recognition (HDR), the efficient and accurate recognition of text against complex backgrounds, diverse handwriting styles, and varying document layouts remains a practical challenge. Moreover, this issue is seldom addressed in academic research, particularly in scenarios with minimal annotated data available. In this paper, we introduce the DocTTT framework to address these challenges. The key innovation of our approach is that it uses test-time training to adapt the model to each specific input during testing. We propose a novel Meta-Auxiliary learning approach that combines Meta-learning and self-supervised Masked Autoencoder~(MAE). During testing, we adapt the visual representation parameters using a self-supervised MAE loss. During training, we learn the model parameters using a meta-learning framework, so that the model parameters are learned to adapt to a new input effectively. Experimental results show that our proposed method significantly outperforms existing state-of-the-art approaches on benchmark datasets.

Related papers

• Improving Applicability of Deep Learning based Token Classification models during Training [0.0]
  We show that classification metrics, represented by the F1-Score, are insufficient for evaluating the applicability of machine learning models in practice. We introduce a novel metric, Document Integrity Precision (DIP), as a solution for visual document understanding and the token classification task.
  arXiv  Detail & Related papers  (2025-03-28T17:01:19Z)
• Training Plug-n-Play Knowledge Modules with Deep Context Distillation [52.94830874557649]
  In this paper, we propose a way of modularizing knowledge by training document-level Knowledge Modules (KMs) KMs are lightweight components implemented as parameter-efficient LoRA modules, which are trained to store information about new documents. Our method outperforms standard next-token prediction and pre-instruction training techniques, across two datasets.
  arXiv  Detail & Related papers  (2025-03-11T01:07:57Z)
• A Hybrid Model for Few-Shot Text Classification Using Transfer and Meta-Learning [0.0]
  This paper proposes a few-shot text classification model based on transfer learning and meta-learning. Under the conditions of few samples and medium samples, the model based on transfer learning and meta-learning significantly outperforms traditional machine learning and deep learning methods.
  arXiv  Detail & Related papers  (2025-02-13T09:00:32Z)
• Adapting Vision-Language Models to Open Classes via Test-Time Prompt Tuning [50.26965628047682]
  Adapting pre-trained models to open classes is a challenging problem in machine learning. In this paper, we consider combining the advantages of both and come up with a test-time prompt tuning approach. Our proposed method outperforms all comparison methods on average considering both base and new classes.
  arXiv  Detail & Related papers  (2024-08-29T12:34:01Z)
• Combating Missing Modalities in Egocentric Videos at Test Time [92.38662956154256]
  Real-world applications often face challenges with incomplete modalities due to privacy concerns, efficiency needs, or hardware issues. We propose a novel approach to address this issue at test time without requiring retraining. MiDl represents the first self-supervised, online solution for handling missing modalities exclusively at test time.
  arXiv  Detail & Related papers  (2024-04-23T16:01:33Z)
• Test-Time Training on Graphs with Large Language Models (LLMs) [68.375487369596]
  Test-Time Training (TTT) has been proposed as a promising approach to train Graph Neural Networks (GNNs) Inspired by the great annotation ability of Large Language Models (LLMs) on Text-Attributed Graphs (TAGs), we propose to enhance the test-time training on graphs with LLMs as annotators. A two-stage training strategy is designed to tailor the test-time model with the limited and noisy labels.
  arXiv  Detail & Related papers  (2024-04-21T08:20:02Z)
• Development of an NLP-driven computer-based test guide for visually impaired students [0.28647133890966986]
  This paper presents an NLP-driven Computer-Based Test guide for visually impaired students. It employs a speech technology pre-trained methods to provide real-time assistance and support to visually impaired students.
  arXiv  Detail & Related papers  (2024-01-22T21:59:00Z)
• Context-Aware Meta-Learning [52.09326317432577]
  We propose a meta-learning algorithm that emulates Large Language Models by learning new visual concepts during inference without fine-tuning. Our approach exceeds or matches the state-of-the-art algorithm, P>M>F, on 8 out of 11 meta-learning benchmarks.
  arXiv  Detail & Related papers  (2023-10-17T03:35:27Z)
• MetricPrompt: Prompting Model as a Relevance Metric for Few-shot Text Classification [65.51149771074944]
  MetricPrompt eases verbalizer design difficulty by reformulating few-shot text classification task into text pair relevance estimation task. We conduct experiments on three widely used text classification datasets across four few-shot settings. Results show that MetricPrompt outperforms manual verbalizer and other automatic verbalizer design methods across all few-shot settings.
  arXiv  Detail & Related papers  (2023-06-15T06:51:35Z)
• CSSL-MHTR: Continual Self-Supervised Learning for Scalable Multi-script Handwritten Text Recognition [16.987008461171065]
  We explore the potential of continual self-supervised learning to alleviate the catastrophic forgetting problem in handwritten text recognition. Our method consists in adding intermediate layers called adapters for each task, and efficiently distilling knowledge from the previous model while learning the current task. We attain state-of-the-art performance on English, Italian and Russian scripts, whilst adding only a few parameters per task.
  arXiv  Detail & Related papers  (2023-03-16T14:27:45Z)
• Unified Pretraining Framework for Document Understanding [52.224359498792836]
  We present UDoc, a new unified pretraining framework for document understanding. UDoc is designed to support most document understanding tasks, extending the Transformer to take multimodal embeddings as input. An important feature of UDoc is that it learns a generic representation by making use of three self-supervised losses.
  arXiv  Detail & Related papers  (2022-04-22T21:47:04Z)
• Robust Document Representations using Latent Topics and Metadata [17.306088038339336]
  We propose a novel approach to fine-tuning a pre-trained neural language model for document classification problems. We generate document representations that capture both text and metadata artifacts in a task manner. Our solution also incorporates metadata explicitly rather than just augmenting them with text.
  arXiv  Detail & Related papers  (2020-10-23T21:52:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.