TextCrafter: Accurately Rendering Multiple Texts in Complex Visual Scenes

• URL: http://arxiv.org/abs/2503.23461v2
• Date: Tue, 01 Apr 2025 02:56:45 GMT
• Title: TextCrafter: Accurately Rendering Multiple Texts in Complex Visual Scenes
• Authors: Nikai Du, Zhennan Chen, Zhizhou Chen, Shan Gao, Xi Chen, Zhengkai Jiang, Jian Yang, Ying Tai,
• Abstract summary: Complex Visual Text Generation (CVTG) centers on generating intricate textual content distributed across diverse regions within visual images.<n>InG, image generation models often rendering distorted and blurred visual text or missing some visual text.<n>We propose TextCrafter, a novel multi-visual text rendering method.
• Score: 30.947323186487885
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper explores the task of Complex Visual Text Generation (CVTG), which centers on generating intricate textual content distributed across diverse regions within visual images. In CVTG, image generation models often rendering distorted and blurred visual text or missing some visual text. To tackle these challenges, we propose TextCrafter, a novel multi-visual text rendering method. TextCrafter employs a progressive strategy to decompose complex visual text into distinct components while ensuring robust alignment between textual content and its visual carrier. Additionally, it incorporates a token focus enhancement mechanism to amplify the prominence of visual text during the generation process. TextCrafter effectively addresses key challenges in CVTG tasks, such as text confusion, omissions, and blurriness. Moreover, we present a new benchmark dataset, CVTG-2K, tailored to rigorously evaluate the performance of generative models on CVTG tasks. Extensive experiments demonstrate that our method surpasses state-of-the-art approaches.

Related papers

• Visual Text Processing: A Comprehensive Review and Unified Evaluation [99.57846940547171]
  We present a comprehensive, multi-perspective analysis of recent advancements in visual text processing. Our aim is to establish this work as a fundamental resource that fosters future exploration and innovation in the dynamic field of visual text processing.
  arXiv  Detail & Related papers  (2025-04-30T14:19:29Z)
• TextInVision: Text and Prompt Complexity Driven Visual Text Generation Benchmark [61.412934963260724]
  Existing diffusion-based text-to-image models often struggle to accurately embed text within images.<n>We introduce TextInVision, a large-scale, text and prompt complexity driven benchmark to evaluate the ability of diffusion models to integrate visual text into images.
  arXiv  Detail & Related papers  (2025-03-17T21:36:31Z)
• Text-Animator: Controllable Visual Text Video Generation [149.940821790235]
  We propose an innovative approach termed Text-Animator for visual text video generation. Text-Animator contains a text embedding injection module to precisely depict the structures of visual text in generated videos. We also develop a camera control module and a text refinement module to improve the stability of generated visual text.
  arXiv  Detail & Related papers  (2024-06-25T17:59:41Z)
• Vision-Aware Text Features in Referring Image Segmentation: From Object Understanding to Context Understanding [26.768147543628096]
  We propose a novel framework that emphasizes object and context comprehension inspired by human cognitive processes. Our method achieves significant performance improvements on three benchmark datasets.
  arXiv  Detail & Related papers  (2024-04-12T16:38:48Z)
• Towards Improving Document Understanding: An Exploration on Text-Grounding via MLLMs [96.54224331778195]
  We present a text-grounding document understanding model, termed TGDoc, which enhances MLLMs with the ability to discern the spatial positioning of text within images. We formulate instruction tuning tasks including text detection, recognition, and spotting to facilitate the cohesive alignment between the visual encoder and large language model. Our method achieves state-of-the-art performance across multiple text-rich benchmarks, validating the effectiveness of our method.
  arXiv  Detail & Related papers  (2023-11-22T06:46:37Z)
• TextFormer: A Query-based End-to-End Text Spotter with Mixed Supervision [61.186488081379]
  We propose TextFormer, a query-based end-to-end text spotter with Transformer architecture. TextFormer builds upon an image encoder and a text decoder to learn a joint semantic understanding for multi-task modeling. It allows for mutual training and optimization of classification, segmentation, and recognition branches, resulting in deeper feature sharing.
  arXiv  Detail & Related papers  (2023-06-06T03:37:41Z)
• Aggregated Text Transformer for Scene Text Detection [5.387121933662753]
  We present the Aggregated Text TRansformer(ATTR), which is designed to represent texts in scene images with a multi-scale self-attention mechanism. The multi-scale image representations are robust and contain rich information on text contents of various sizes. The proposed method detects scene texts by representing each text instance as an individual binary mask, which is tolerant of curve texts and regions with dense instances.
  arXiv  Detail & Related papers  (2022-11-25T09:47:34Z)
• Language Matters: A Weakly Supervised Pre-training Approach for Scene Text Detection and Spotting [69.77701325270047]
  This paper presents a weakly supervised pre-training method that can acquire effective scene text representations. Our network consists of an image encoder and a character-aware text encoder that extract visual and textual features. Experiments show that our pre-trained model improves F-score by +2.5% and +4.8% while transferring its weights to other text detection and spotting networks.
  arXiv  Detail & Related papers  (2022-03-08T08:10:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.