Multi-Modal Character Localization and Extraction for Chinese Text Recognition

• URL: http://arxiv.org/abs/2603.13886v1
• Date: Sat, 14 Mar 2026 10:53:47 GMT
• Title: Multi-Modal Character Localization and Extraction for Chinese Text Recognition
• Abstract summary: We propose a novel method named LER, which explicitly decouples each character and independently recognizes characters.<n>LER takes into account the complex inner structures of Chinese to provide the text prediction results.<n>Experiments conducted on six English benchmarks and the Union14M benchmark show impressive results in English text recognition by LER.
• Score: 2.9818717410831392
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Scene text recognition (STR) methods have demonstrated their excellent capability in English text images. However, due to the complex inner structures of Chinese and the extensive character categories, it poses challenges for recognizing Chinese text in images. Recently, studies have shown that the methods designed for English text recognition encounter an accuracy bottleneck when recognizing Chinese text images. This raises the question: Is it appropriate to apply the model developed for English to the Chinese STR task? To explore this issue, we propose a novel method named LER, which explicitly decouples each character and independently recognizes characters while taking into account the complex inner structures of Chinese. LER consists of three modules: Localization, Extraction, and Recognition. Firstly, the localization module utilizes multimodal information to determine the character's position precisely. Then, the extraction module dissociates all characters in parallel. Finally, the recognition module considers the unique inner structures of Chinese to provide the text prediction results. Extensive experiments conducted on large-scale Chinese benchmarks indicate that our method significantly outperforms existing methods. Furthermore, extensive experiments conducted on six English benchmarks and the Union14M benchmark show impressive results in English text recognition by LER. Code is available at https://github.com/Pandarenlql/LER.

Related papers

• Beyond Cropped Regions: New Benchmark and Corresponding Baseline for Chinese Scene Text Retrieval in Diverse Layouts [2.9100667158464035]
  Chinese scene text retrieval is a practical task that aims to search for images containing visual instances of a Chinese query text.<n>Current efforts tend to inherit the solution for English scene text retrieval, failing to achieve satisfactory performance.<n>We propose Chinese Scene Text Retrieval CLIP (CSTR-CLIP), a novel model that integrates global visual information with multi-granularity alignment training.
  arXiv  Detail & Related papers  (2025-06-05T13:10:17Z)
• Zero-Shot Chinese Character Recognition with Hierarchical Multi-Granularity Image-Text Aligning [52.92837273570818]
  Chinese characters exhibit unique structures and compositional rules, allowing for the use of fine-grained semantic information in representation.<n>We propose a Hierarchical Multi-Granularity Image-Text Aligning (Hi-GITA) framework based on a contrastive paradigm.<n>Our proposed Hi-GITA outperforms existing zero-shot CCR methods.
  arXiv  Detail & Related papers  (2025-05-30T17:39:14Z)
• HierCode: A Lightweight Hierarchical Codebook for Zero-shot Chinese Text Recognition [47.86479271322264]
  We propose HierCode, a novel and lightweight codebook that exploits the innate hierarchical nature of Chinese characters. HierCode employs a multi-hot encoding strategy, leveraging hierarchical binary tree encoding and prototype learning to create distinctive, informative representations for each character. This approach not only facilitates zero-shot recognition of OOV characters by utilizing shared radicals and structures but also excels in line-level recognition tasks by computing similarity with visual features.
  arXiv  Detail & Related papers  (2024-03-20T17:20:48Z)
• Decomposed Prompting: Probing Multilingual Linguistic Structure Knowledge in Large Language Models [54.58989938395976]
  We introduce a decomposed prompting approach for sequence labeling tasks.<n>We test our method on the Universal Dependencies part-of-speech tagging dataset for 38 languages.
  arXiv  Detail & Related papers  (2024-02-28T15:15:39Z)
• Chinese Text Recognition with A Pre-Trained CLIP-Like Model Through Image-IDS Aligning [61.34060587461462]
  We propose a two-stage framework for Chinese Text Recognition (CTR) We pre-train a CLIP-like model through aligning printed character images and Ideographic Description Sequences (IDS) This pre-training stage simulates humans recognizing Chinese characters and obtains the canonical representation of each character. The learned representations are employed to supervise the CTR model, such that traditional single-character recognition can be improved to text-line recognition.
  arXiv  Detail & Related papers  (2023-09-03T05:33:16Z)
• Orientation-Independent Chinese Text Recognition in Scene Images [61.34060587461462]
  We take the first attempt to extract orientation-independent visual features by disentangling content and orientation information of text images. Specifically, we introduce a Character Image Reconstruction Network (CIRN) to recover corresponding printed character images with disentangled content and orientation information.
  arXiv  Detail & Related papers  (2023-09-03T05:30:21Z)
• A Benchmark for Chinese-English Scene Text Image Super-resolution [15.042152725255171]
  Scene Text Image Super-resolution (STISR) aims to recover high-resolution (HR) scene text images with visually pleasant and readable text content from low-resolution (LR) input. Most existing works focus on recovering English texts, which have relatively simple character structures. We propose a real-world Chinese-English benchmark dataset, namely Real-CE, for the task of STISR.
  arXiv  Detail & Related papers  (2023-08-07T02:57:48Z)
• TextFormer: A Query-based End-to-End Text Spotter with Mixed Supervision [61.186488081379]
  We propose TextFormer, a query-based end-to-end text spotter with Transformer architecture. TextFormer builds upon an image encoder and a text decoder to learn a joint semantic understanding for multi-task modeling. It allows for mutual training and optimization of classification, segmentation, and recognition branches, resulting in deeper feature sharing.
  arXiv  Detail & Related papers  (2023-06-06T03:37:41Z)
• Benchmarking Chinese Text Recognition: Datasets, Baselines, and an Empirical Study [25.609450020149637]
  Existing text recognition methods are mainly for English texts, whereas ignoring the pivotal role of Chinese texts. We manually collect Chinese text datasets from publicly available competitions, projects, and papers, then divide them into four categories including scene, web, document, and handwriting datasets. By analyzing the experimental results, we surprisingly observe that state-of-the-art baselines for recognizing English texts cannot perform well on Chinese scenarios.
  arXiv  Detail & Related papers  (2021-12-30T15:30:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.