Accurate Scene Text Recognition with Efficient Model Scaling and Cloze Self-Distillation

• URL: http://arxiv.org/abs/2503.16184v1
• Date: Thu, 20 Mar 2025 14:35:46 GMT
• Title: Accurate Scene Text Recognition with Efficient Model Scaling and Cloze Self-Distillation
• Authors: Andrea Maracani, Savas Ozkan, Sijun Cho, Hyowon Kim, Eunchung Noh, Jeongwon Min, Cho Jung Min, Dookun Park, Mete Ozay,
• Abstract summary: We show that scaling the decoder yields significant performance gains, always exceeding those achieved by encoder scaling alone.<n>We also identify label noise as a key challenge in Scene Text Recognition, particularly in real-world data.<n>Our methodology achieves state-of-the-art performance on 10 out of 11 benchmarks using only real data, while significantly reducing the parameter size and computational costs.
• Score: 11.217033010884006
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Scaling architectures have been proven effective for improving Scene Text Recognition (STR), but the individual contribution of vision encoder and text decoder scaling remain under-explored. In this work, we present an in-depth empirical analysis and demonstrate that, contrary to previous observations, scaling the decoder yields significant performance gains, always exceeding those achieved by encoder scaling alone. We also identify label noise as a key challenge in STR, particularly in real-world data, which can limit the effectiveness of STR models. To address this, we propose Cloze Self-Distillation (CSD), a method that mitigates label noise by distilling a student model from context-aware soft predictions and pseudolabels generated by a teacher model. Additionally, we enhance the decoder architecture by introducing differential cross-attention for STR. Our methodology achieves state-of-the-art performance on 10 out of 11 benchmarks using only real data, while significantly reducing the parameter size and computational costs.

Related papers

• Efficient and Accurate Scene Text Recognition with Cascaded-Transformers [11.638859439061164]
  We propose an efficient and accurate Scene Text Recognition system. We focus on improving the efficiency of encoder models by introducing a cascaded-transformers structure. Our experimental results confirm that our STR system achieves comparable performance to state-of-the-art baselines.
  arXiv  Detail & Related papers  (2025-03-24T16:58:37Z)
• "Principal Components" Enable A New Language of Images [79.45806370905775]
  We introduce a novel visual tokenization framework that embeds a provable PCA-like structure into the latent token space. Our approach achieves state-of-the-art reconstruction performance and enables better interpretability to align with the human vision system.
  arXiv  Detail & Related papers  (2025-03-11T17:59:41Z)
• Robust and Efficient Writer-Independent IMU-Based Handwriting Recognization [2.022040536958726]
  Online handwriting recognition (HWR) using data from inertial measurement units (IMUs) remains challenging.<n>Traditional models often struggle to recognize handwriting from unseen writers.<n>This paper presents an HWR model with an encoder-decoder structure for IMU data, featuring a CNN-based encoder for feature extraction and a BiLSTM decoder for sequence modeling.
  arXiv  Detail & Related papers  (2025-02-28T11:09:28Z)
• Leveraging Semi-Supervised Learning to Enhance Data Mining for Image Classification under Limited Labeled Data [35.431340001608476]
  Traditional data mining methods are inadequate when faced with large-scale, high-dimensional and complex data.<n>This study introduces semi-supervised learning methods, aiming to improve the algorithm's ability to utilize unlabeled data.<n> Specifically, we adopt a self-training method and combine it with a convolutional neural network (CNN) for image feature extraction and classification.
  arXiv  Detail & Related papers  (2024-11-27T18:59:50Z)
• IncSAR: A Dual Fusion Incremental Learning Framework for SAR Target Recognition [13.783950035836593]
  IncSAR is an incremental learning framework designed to tackle catastrophic forgetting in target recognition.<n>To mitigate the speckle noise inherent in SAR images, we employ a denoising module based on a neural network approximation.<n>Experiments on the MSTAR, SAR-AIRcraft-1.0, and OpenSARShip benchmark datasets demonstrate that IncSAR significantly outperforms state-of-the-art approaches.
  arXiv  Detail & Related papers  (2024-10-08T08:49:47Z)
• Self-Taught Recognizer: Toward Unsupervised Adaptation for Speech Foundation Models [84.8919069953397]
  Self-TAught Recognizer (STAR) is an unsupervised adaptation framework for speech recognition systems. We show that STAR achieves an average of 13.5% relative reduction in word error rate across 14 target domains. STAR exhibits high data efficiency that only requires less than one-hour unlabeled data.
  arXiv  Detail & Related papers  (2024-05-23T04:27:11Z)
• Fine tuning Pre trained Models for Robustness Under Noisy Labels [34.68018860186995]
  The presence of noisy labels in a training dataset can significantly impact the performance of machine learning models. We introduce a novel algorithm called TURN, which robustly and efficiently transfers the prior knowledge of pre-trained models.
  arXiv  Detail & Related papers  (2023-10-24T20:28:59Z)
• A Discrepancy Aware Framework for Robust Anomaly Detection [51.710249807397695]
  We present a Discrepancy Aware Framework (DAF), which demonstrates robust performance consistently with simple and cheap strategies. Our method leverages an appearance-agnostic cue to guide the decoder in identifying defects, thereby alleviating its reliance on synthetic appearance. Under the simple synthesis strategies, it outperforms existing methods by a large margin. Furthermore, it also achieves the state-of-the-art localization performance.
  arXiv  Detail & Related papers  (2023-10-11T15:21:40Z)
• Improved Speech Emotion Recognition using Transfer Learning and Spectrogram Augmentation [56.264157127549446]
  Speech emotion recognition (SER) is a challenging task that plays a crucial role in natural human-computer interaction. One of the main challenges in SER is data scarcity. We propose a transfer learning strategy combined with spectrogram augmentation.
  arXiv  Detail & Related papers  (2021-08-05T10:39:39Z)
• SCARF: Self-Supervised Contrastive Learning using Random Feature Corruption [72.35532598131176]
  We propose SCARF, a technique for contrastive learning, where views are formed by corrupting a random subset of features. We show that SCARF complements existing strategies and outperforms alternatives like autoencoders.
  arXiv  Detail & Related papers  (2021-06-29T08:08:33Z)
• SoundCLR: Contrastive Learning of Representations For Improved Environmental Sound Classification [0.6767885381740952]
  SoundCLR is a supervised contrastive learning method for effective environment sound classification with state-of-the-art performance. Due to the comparatively small sizes of the available environmental sound datasets, we propose and exploit a transfer learning and strong data augmentation pipeline. Our experiments show that our masking based augmentation technique on the log-mel spectrograms can significantly improve the recognition performance.
  arXiv  Detail & Related papers  (2021-03-02T18:42:45Z)
• A Self-Refinement Strategy for Noise Reduction in Grammatical Error Correction [54.569707226277735]
  Existing approaches for grammatical error correction (GEC) rely on supervised learning with manually created GEC datasets. There is a non-negligible amount of "noise" where errors were inappropriately edited or left uncorrected. We propose a self-refinement method where the key idea is to denoise these datasets by leveraging the prediction consistency of existing models.
  arXiv  Detail & Related papers  (2020-10-07T04:45:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.