TextSculptor: Training and Benchmarking Scene Text Editing

• URL: http://arxiv.org/abs/2605.21090v1
• Date: Wed, 20 May 2026 12:22:26 GMT
• Title: TextSculptor: Training and Benchmarking Scene Text Editing
• Abstract summary: We present TextSculptor, a comprehensive framework for data construction and evaluation of scene text editing.<n>TextSculptor improves open-source text editing performance and narrows the gap to proprietary models.
• Score: 88.11688559021628
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent advances in Multimodal Large Language Models (MLLMs) and diffusion-based generative models have substantially improved prompt-driven image editing. However, scene text editing remains challenging, as it requires models to precisely modify textual content while preserving visual realism and non-target regions. Current open-source models still lag behind proprietary systems, largely due to the scarcity of high-quality training data and the lack of standardized benchmarks tailored to text editing. To address these challenges, we present TextSculptor, a comprehensive framework for data construction and evaluation of scene text editing. We first develop an automated data construction pipeline that combines text-aware image synthesis with programmatic text rendering and compositing. Based on this pipeline, we build TextSculpt-Data, a large-scale dataset containing 3.2M training samples, including 1.2M OCR-verified text-to-image samples and 2M paired text editing samples with naturally aligned source-target images and strong background consistency. We further introduce TextSculpt-Bench, a benchmark covering four fundamental text editing tasks: text addition, text replacement, text removal, and hybrid editing. To support reliable evaluation, we design a tailored protocol that measures text accuracy, visual quality, and background preservation through OCR-based text alignment, multimodal judgment, and background-region similarity. Extensive experiments show that TextSculptor improves open-source text editing performance and narrows the gap to proprietary models. The data and benchmark are available at https://github.com/linyiheng123/TextSculptor.

Related papers

• TextWand: A Unified Framework for Scene Text Editing [32.59846554728947]
  TextWand is a framework that unifies scene text removal, generation, and replacement into a single model.<n>It achieves precise control over both text appearance and background integrity.<n>It delivers superior text content accuracy, layout and style consistency, and overall image quality across scene text removal, generation and replacement tasks.
  arXiv  Detail & Related papers  (2026-06-04T05:43:24Z)
• WeEdit: A Dataset, Benchmark and Glyph-Guided Framework for Text-centric Image Editing [103.68419705566146]
  We present WeEdit, a systematic solution encompassing a scalable data construction pipeline, two benchmarks, and a tailored two-stage training strategy.<n>Specifically, we propose a novel HTML-based automatic editing pipeline, which generates 330K training pairs covering diverse editing operations and 15 languages.<n>On the algorithmic side, we employ glyph-guided supervised fine-tuning to inject explicit spatial and content priors, followed by a multi-objective reinforcement learning stage to align generation with instruction adherence, text clarity, and background preservation.
  arXiv  Detail & Related papers  (2026-03-12T06:25:09Z)
• TextGuider: Training-Free Guidance for Text Rendering via Attention Alignment [68.91073792449201]
  We propose TextGuider, a training-free method that encourages accurate and complete text appearance.<n>Specifically, we analyze attention patterns in Multi-Modal Diffusion Transformer(MM-DiT) models, particularly for text-related tokens intended to be rendered in the image.<n>Our method achieves state-of-the-art performance in test-time text rendering, with significant gains in recall and strong results in OCR accuracy and CLIP score.
  arXiv  Detail & Related papers  (2025-12-10T06:18:30Z)
• STELLAR: Scene Text Editor for Low-Resource Languages and Real-World Data [3.622341086373503]
  We propose STELLAR, a Scene Text Editor for Low-resource LAnguages and Real-world data.<n> STELLAR enables reliable multilingual editing through a language-adaptive glyph encoder and a multi-stage training strategy.<n>We also construct a new dataset, STIPLAR(Scene Text Image Pairs of Low-resource lAnguages and Real-world data), for training and evaluation.
  arXiv  Detail & Related papers  (2025-11-13T05:18:03Z)
• TextMastero: Mastering High-Quality Scene Text Editing in Diverse Languages and Styles [12.182588762414058]
  Scene text editing aims to modify texts on images while maintaining the style of newly generated text similar to the original. Recent works leverage diffusion models, showing improved results, yet still face challenges. We present emphTextMastero - a carefully designed multilingual scene text editing architecture based on latent diffusion models (LDMs)
  arXiv  Detail & Related papers  (2024-08-20T08:06:09Z)
• Enhancing Scene Text Detectors with Realistic Text Image Synthesis Using Diffusion Models [63.99110667987318]
  We present DiffText, a pipeline that seamlessly blends foreground text with the background's intrinsic features. With fewer text instances, our produced text images consistently surpass other synthetic data in aiding text detectors.
  arXiv  Detail & Related papers  (2023-11-28T06:51:28Z)
• AnyText: Multilingual Visual Text Generation And Editing [18.811943975513483]
  We introduce AnyText, a diffusion-based multilingual visual text generation and editing model. AnyText can write characters in multiple languages, to the best of our knowledge, this is the first work to address multilingual visual text generation. We contribute the first large-scale multilingual text images dataset, AnyWord-3M, containing 3 million image-text pairs with OCR annotations in multiple languages.
  arXiv  Detail & Related papers  (2023-11-06T12:10:43Z)
• TextFormer: A Query-based End-to-End Text Spotter with Mixed Supervision [61.186488081379]
  We propose TextFormer, a query-based end-to-end text spotter with Transformer architecture. TextFormer builds upon an image encoder and a text decoder to learn a joint semantic understanding for multi-task modeling. It allows for mutual training and optimization of classification, segmentation, and recognition branches, resulting in deeper feature sharing.
  arXiv  Detail & Related papers  (2023-06-06T03:37:41Z)
• TextDiffuser: Diffusion Models as Text Painters [118.30923824681642]
  We introduce TextDiffuser, focusing on generating images with visually appealing text that is coherent with backgrounds. We contribute the first large-scale text images dataset with OCR annotations, MARIO-10M, containing 10 million image-text pairs. We show that TextDiffuser is flexible and controllable to create high-quality text images using text prompts alone or together with text template images, and conduct text inpainting to reconstruct incomplete images with text.
  arXiv  Detail & Related papers  (2023-05-18T10:16:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.