Focus on the Whole Character: Discriminative Character Modeling for Scene Text Recognition

• URL: http://arxiv.org/abs/2407.05562v1
• Date: Mon, 8 Jul 2024 02:33:29 GMT
• Title: Focus on the Whole Character: Discriminative Character Modeling for Scene Text Recognition
• Authors: Bangbang Zhou, Yadong Qu, Zixiao Wang, Zicheng Li, Boqiang Zhang, Hongtao Xie,
• Abstract summary: We propose a novel method that enriches the character features to enhance discriminability of characters. CACE introduces a decay matrix in each block to explicitly guide the attention region for each token. I2CL improves the discnative capability of features by learning a long-term memory unit for each character category.
• Score: 28.93482989766411
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recently, scene text recognition (STR) models have shown significant performance improvements. However, existing models still encounter difficulties in recognizing challenging texts that involve factors such as severely distorted and perspective characters. These challenging texts mainly cause two problems: (1) Large Intra-Class Variance. (2) Small Inter-Class Variance. An extremely distorted character may prominently differ visually from other characters within the same category, while the variance between characters from different classes is relatively small. To address the above issues, we propose a novel method that enriches the character features to enhance the discriminability of characters. Firstly, we propose the Character-Aware Constraint Encoder (CACE) with multiple blocks stacked. CACE introduces a decay matrix in each block to explicitly guide the attention region for each token. By continuously employing the decay matrix, CACE enables tokens to perceive morphological information at the character level. Secondly, an Intra-Inter Consistency Loss (I^2CL) is introduced to consider intra-class compactness and inter-class separability at feature space. I^2CL improves the discriminative capability of features by learning a long-term memory unit for each character category. Trained with synthetic data, our model achieves state-of-the-art performance on common benchmarks (94.1% accuracy) and Union14M-Benchmark (61.6% accuracy). Code is available at https://github.com/bang123-box/CFE.

Related papers

• DISC: Plug-and-Play Decoding Intervention with Similarity of Characters for Chinese Spelling Check [37.44133266050293]
  We propose a light-weight plug-and-play DISC (i.e., decoding intervention with similarity of characters) module for Chinese spelling check (CSC) models. DISC measures phonetic and glyph similarities between characters and incorporates this similarity information only during the inference phase. Experiments on three CSC benchmarks demonstrate that our proposed method significantly improves model performance, approaching and even surpassing the current state-of-the-art models.
  arXiv  Detail & Related papers  (2024-12-17T12:44:06Z)
• Boosting Semi-Supervised Scene Text Recognition via Viewing and Summarizing [71.29488677105127]
  Existing scene text recognition (STR) methods struggle to recognize challenging texts, especially for artistic and severely distorted characters. We propose a contrastive learning-based STR framework by leveraging synthetic and real unlabeled data without any human cost. Our method achieves SOTA performance (94.7% and 70.9% average accuracy on common benchmarks and Union14M-Benchmark.
  arXiv  Detail & Related papers  (2024-11-23T15:24:47Z)
• KNN Transformer with Pyramid Prompts for Few-Shot Learning [52.735070934075736]
  Few-Shot Learning aims to recognize new classes with limited labeled data. Recent studies have attempted to address the challenge of rare samples with textual prompts to modulate visual features.
  arXiv  Detail & Related papers  (2024-10-14T07:39:30Z)
• Finetuning CLIP to Reason about Pairwise Differences [52.028073305958074]
  We propose an approach to train vision-language models such as CLIP in a contrastive manner to reason about differences in embedding space. We first demonstrate that our approach yields significantly improved capabilities in ranking images by a certain attribute. We also illustrate that the resulting embeddings obey a larger degree of geometric properties in embedding space.
  arXiv  Detail & Related papers  (2024-09-15T13:02:14Z)
• MARS: Paying more attention to visual attributes for text-based person search [6.438244172631555]
  This paper presents a novel TBPS architecture named MARS (Mae-Attribute-Relation-Sensitive) It enhances current state-of-the-art models by introducing two key components: a Visual Reconstruction Loss and an Attribute Loss. Experiments on three commonly used datasets, namely CUHK-PEDES, ICFG-PEDES, and RSTPReid, report performance improvements.
  arXiv  Detail & Related papers  (2024-07-05T06:44:43Z)
• C-LLM: Learn to Check Chinese Spelling Errors Character by Character [61.53865964535705]
  We propose C-LLM, a Large Language Model-based Chinese Spell Checking method that learns to check errors Character by Character. C-LLM achieves an average improvement of 10% over existing methods.
  arXiv  Detail & Related papers  (2024-06-24T11:16:31Z)
• Deep Learning-Driven Approach for Handwritten Chinese Character Classification [0.0]
  Handwritten character recognition is a challenging problem for machine learning researchers. With numerous unique character classes present, some data, such as Logographic Scripts or Sino-Korean character sequences, bring new complications to the HCR problem. This paper proposes a highly scalable approach for detailed character image classification by introducing the model architecture, data preprocessing steps, and testing design instructions.
  arXiv  Detail & Related papers  (2024-01-30T15:29:32Z)
• TagCLIP: Improving Discrimination Ability of Open-Vocabulary Semantic Segmentation [53.974228542090046]
  Contrastive Language-Image Pre-training (CLIP) has recently shown great promise in pixel-level zero-shot learning tasks. Existing approaches utilizing CLIP's text and patch embeddings to generate semantic masks often misidentify input pixels from unseen classes. We propose TagCLIP (Trusty-aware guided CLIP) to address this issue.
  arXiv  Detail & Related papers  (2023-04-15T12:52:23Z)
• CDistNet: Perceiving Multi-Domain Character Distance for Robust Text Recognition [87.3894423816705]
  We propose a novel module called Multi-Domain Character Distance Perception (MDCDP) to establish a visually and semantically related position embedding. MDCDP uses the position embedding to query both visual and semantic features following the cross-attention mechanism. We develop CDistNet that stacks multiple MDCDPs to guide a gradually precise distance modeling.
  arXiv  Detail & Related papers  (2021-11-22T06:27:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.