Towards Robust Real-Time Scene Text Detection: From Semantic to Instance Representation Learning

• URL: http://arxiv.org/abs/2308.07202v1
• Date: Mon, 14 Aug 2023 15:14:37 GMT
• Title: Towards Robust Real-Time Scene Text Detection: From Semantic to Instance Representation Learning
• Authors: Xugong Qin, Pengyuan Lyu, Chengquan Zhang, Yu Zhou, Kun Yao, Peng Zhang, Hailun Lin, Weiping Wang
• Abstract summary: We propose representation learning for real-time scene text detection. For semantic representation learning, we propose global-dense semantic contrast (GDSC) and top-down modeling (TDM) With the proposed GDSC and TDM, the encoder network learns stronger representation without introducing any parameters and computations during inference. The proposed method achieves 87.2% F-measure with 48.2 FPS on Total-Text and 89.6% F-measure with 36.9 FPS on MSRA-TD500.
• Score: 19.856492291263102
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Due to the flexible representation of arbitrary-shaped scene text and simple pipeline, bottom-up segmentation-based methods begin to be mainstream in real-time scene text detection. Despite great progress, these methods show deficiencies in robustness and still suffer from false positives and instance adhesion. Different from existing methods which integrate multiple-granularity features or multiple outputs, we resort to the perspective of representation learning in which auxiliary tasks are utilized to enable the encoder to jointly learn robust features with the main task of per-pixel classification during optimization. For semantic representation learning, we propose global-dense semantic contrast (GDSC), in which a vector is extracted for global semantic representation, then used to perform element-wise contrast with the dense grid features. To learn instance-aware representation, we propose to combine top-down modeling (TDM) with the bottom-up framework to provide implicit instance-level clues for the encoder. With the proposed GDSC and TDM, the encoder network learns stronger representation without introducing any parameters and computations during inference. Equipped with a very light decoder, the detector can achieve more robust real-time scene text detection. Experimental results on four public datasets show that the proposed method can outperform or be comparable to the state-of-the-art on both accuracy and speed. Specifically, the proposed method achieves 87.2% F-measure with 48.2 FPS on Total-Text and 89.6% F-measure with 36.9 FPS on MSRA-TD500 on a single GeForce RTX 2080 Ti GPU.

Related papers

• Adaptive Segmentation Network for Scene Text Detection [0.0]
  We propose to automatically learn the discriminate segmentation threshold, which distinguishes text pixels from background pixels for segmentation-based scene text detectors. Besides, we design a Global-information Enhanced Feature Pyramid Network (GE-FPN) for capturing text instances with macro size and extreme aspect ratios. Finally, together with the proposed threshold learning strategy and text detection structure, we design an Adaptive Network (ASNet) for scene text detection.
  arXiv  Detail & Related papers  (2023-07-27T17:37:56Z)
• LRANet: Towards Accurate and Efficient Scene Text Detection with Low-Rank Approximation Network [63.554061288184165]
  We propose a novel parameterized text shape method based on low-rank approximation. By exploring the shape correlation among different text contours, our method achieves consistency, compactness, simplicity, and robustness in shape representation. We implement an accurate and efficient arbitrary-shaped text detector named LRANet.
  arXiv  Detail & Related papers  (2023-06-27T02:03:46Z)
• TextFormer: A Query-based End-to-End Text Spotter with Mixed Supervision [61.186488081379]
  We propose TextFormer, a query-based end-to-end text spotter with Transformer architecture. TextFormer builds upon an image encoder and a text decoder to learn a joint semantic understanding for multi-task modeling. It allows for mutual training and optimization of classification, segmentation, and recognition branches, resulting in deeper feature sharing.
  arXiv  Detail & Related papers  (2023-06-06T03:37:41Z)
• UATVR: Uncertainty-Adaptive Text-Video Retrieval [90.8952122146241]
  A common practice is to transfer text-video pairs to the same embedding space and craft cross-modal interactions with certain entities. We propose an Uncertainty-language Text-Video Retrieval approach, termed UATVR, which models each look-up as a distribution matching procedure.
  arXiv  Detail & Related papers  (2023-01-16T08:43:17Z)
• SFNet: Faster and Accurate Semantic Segmentation via Semantic Flow [88.97790684009979]
  A common practice to improve the performance is to attain high-resolution feature maps with strong semantic representation. We propose a Flow Alignment Module (FAM) to learn textitSemantic Flow between feature maps of adjacent levels. We also present a novel Gated Dual Flow Alignment Module to directly align high-resolution feature maps and low-resolution feature maps.
  arXiv  Detail & Related papers  (2022-07-10T08:25:47Z)
• Real-Time Scene Text Detection with Differentiable Binarization and Adaptive Scale Fusion [62.269219152425556]
  segmentation-based scene text detection methods have drawn extensive attention in the scene text detection field. We propose a Differentiable Binarization (DB) module that integrates the binarization process into a segmentation network. An efficient Adaptive Scale Fusion (ASF) module is proposed to improve the scale robustness by fusing features of different scales adaptively.
  arXiv  Detail & Related papers  (2022-02-21T15:30:14Z)
• CentripetalText: An Efficient Text Instance Representation for Scene Text Detection [19.69057252363207]
  We propose an efficient text instance representation named CentripetalText (CT) CT decomposes text instances into the combination of text kernels and centripetal shifts. For the task of scene text detection, our approach achieves superior or competitive performance compared to other existing methods.
  arXiv  Detail & Related papers  (2021-07-13T09:34:18Z)
• RSCA: Real-time Segmentation-based Context-Aware Scene Text Detection [14.125634725954848]
  We propose RSCA: a Real-time-based Context-Aware model for arbitrary-shaped scene text detection. Based on these strategies, RSCA achieves state-of-the-art performance in both speed and accuracy, without complex label assignments or repeated feature aggregations.
  arXiv  Detail & Related papers  (2021-05-26T18:43:17Z)
• PAN++: Towards Efficient and Accurate End-to-End Spotting of Arbitrarily-Shaped Text [85.7020597476857]
  We propose an end-to-end text spotting framework, termed PAN++, which can efficiently detect and recognize text of arbitrary shapes in natural scenes. PAN++ is based on the kernel representation that reformulates a text line as a text kernel (central region) surrounded by peripheral pixels. As a pixel-based representation, the kernel representation can be predicted by a single fully convolutional network, which is very friendly to real-time applications.
  arXiv  Detail & Related papers  (2021-05-02T07:04:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.