TransAnyText: Translating Arbitrary Text in E-commerce Images via Structured Visual Generation

• URL: http://arxiv.org/abs/2608.16284v1
• Date: Mon, 17 Aug 2026 08:54:14 GMT
• Title: TransAnyText: Translating Arbitrary Text in E-commerce Images via Structured Visual Generation
• Abstract summary: Cross-border e-commerce image translation is essential for global retail.<n>We introduce TransAnyText, a structured visual code framework that reformulates image text translation.<n>Our framework generates renderable HTML patches from source images and target languages.
• Score: 27.357240730388014
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Cross-border e-commerce image translation is essential for global retail, where product images, banners, and detail pages need to be produced in different languages. Existing methods struggle to achieve accurate translation, faithful visual identity preservation, and easy-to-edit outputs, simultaneously. To address these challenges, we introduce TransAnyText, a structured visual code framework that reformulates image text translation as generating renderable HTML patches from source images and target languages. Our framework decouples semantic generation from pixel rendering: a vision-language model (VLM) handles visual understanding, cross-lingual translation, and structured visual generation, while a diffusion model performs background inpainting and pixel-level refinement, followed by deterministic rendering to synthesize the final image. Based on this formulation, we develop a three-stage post-training framework, where supervised fine-tuning (SFT) establishes the image-to-code mapping, privilege-gap weighted self-distillation (PWSD) improves the learning of style and layout tokens, and reinforcement learning with verifiable rewards (RLVR) further optimizes task-level performance. We further introduce TransAnyDataset and TransAnyBench, a multilingual dataset and benchmark for e-commerce image translation. Extensive experiments demonstrate competitive performance against cascaded pipelines, open-source end-to-end models, and closed-source image editing systems, providing an effective, controllable, and editable solution for cross-border e-commerce image translation.

Related papers

• UniTranslator: A Unified Multi-modal Framework for End-to-end In-Image Machine Translation [23.787128107000374]
  In-Image Machine Translation (IIMT) aims to translate scene text in an image and render the translated text back into the original regions while preserving the overall visual appearance.<n>Recent unified multimodal models provide a promising solution by combining visual-text understanding and image generation within a single framework.<n>We present UniTranslator, a unified multimodal framework for IIMT that tightly couples translation understanding and text editing.
  arXiv  Detail & Related papers  (2026-06-23T09:11:24Z)
• Images in Sentences: Scaling Interleaved Instructions for Unified Visual Generation [19.404516863003057]
  INSET is a unified generation model that seamlessly embeds images as native vocabulary within textual instructions.<n>We introduce a scalable data engine that synthesizes 15M high-quality interleaved samples from standard image and video datasets.<n>Results on InterleaveBench demonstrate that INSET significantly outperforms state-of-the-art methods in multi-image consistency and text alignment.
  arXiv  Detail & Related papers  (2026-05-12T15:54:49Z)
• Image-to-Image Translation with Diffusion Transformers and CLIP-Based Image Conditioning [2.9603070411207644]
  Diffusion Transformers (DiT) is a diffusion-based framework for image-to-image translation.<n>DiT combines the denoising capabilities of diffusion models with the global modeling power of transformers.<n>We validate our approach on two benchmark datasets: face2comics, which translates real human faces to comic-style illustrations, and edges2shoes, which translates edge maps to realistic shoe images.
  arXiv  Detail & Related papers  (2025-05-21T20:37:33Z)
• Seedream 2.0: A Native Chinese-English Bilingual Image Generation Foundation Model [69.09404597939744]
  Seedream 2.0 is a native Chinese-English bilingual image generation foundation model.<n>It adeptly manages text prompt in both Chinese and English, supporting bilingual image generation and text rendering.<n>It is integrated with a self-developed bilingual large language model as a text encoder, allowing it to learn native knowledge directly from massive data.
  arXiv  Detail & Related papers  (2025-03-10T17:58:33Z)
• Towards Visual Text Design Transfer Across Languages [49.78504488452978]
  We introduce a novel task of Multimodal Style Translation (MuST-Bench) MuST-Bench is a benchmark designed to evaluate the ability of visual text generation models to perform translation across different writing systems. In response, we introduce SIGIL, a framework for multimodal style translation that eliminates the need for style descriptions.
  arXiv  Detail & Related papers  (2024-10-24T15:15:01Z)
• Translatotron-V(ison): An End-to-End Model for In-Image Machine Translation [81.45400849638347]
  In-image machine translation (IIMT) aims to translate an image containing texts in source language into an image containing translations in target language. In this paper, we propose an end-to-end IIMT model consisting of four modules. Our model achieves competitive performance compared to cascaded models with only 70.9% of parameters, and significantly outperforms the pixel-level end-to-end IIMT model.
  arXiv  Detail & Related papers  (2024-07-03T08:15:39Z)
• AnyTrans: Translate AnyText in the Image with Large Scale Models [88.5887934499388]
  This paper introduces AnyTrans, an all-encompassing framework for the task-Translate AnyText in the Image (TATI) Our framework incorporates contextual cues from both textual and visual elements during translation. We have meticulously compiled a test dataset called MTIT6, which consists of multilingual text image translation data from six language pairs.
  arXiv  Detail & Related papers  (2024-06-17T11:37:48Z)
• Unified Language-Vision Pretraining in LLM with Dynamic Discrete Visual Tokenization [52.935150075484074]
  We introduce a well-designed visual tokenizer to translate the non-linguistic image into a sequence of discrete tokens like a foreign language. The resulting visual tokens encompass high-level semantics worthy of a word and also support dynamic sequence length varying from the image. This unification empowers LaVIT to serve as an impressive generalist interface to understand and generate multi-modal content simultaneously.
  arXiv  Detail & Related papers  (2023-09-09T03:01:38Z)
• Generating Images with Multimodal Language Models [78.6660334861137]
  We propose a method to fuse frozen text-only large language models with pre-trained image encoder and decoder models. Our model demonstrates a wide suite of multimodal capabilities: image retrieval, novel image generation, and multimodal dialogue.
  arXiv  Detail & Related papers  (2023-05-26T19:22:03Z)
• Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translation [10.39028769374367]
  We present a new framework that takes text-to-image synthesis to the realm of image-to-image translation. Our method harnesses the power of a pre-trained text-to-image diffusion model to generate a new image that complies with the target text.
  arXiv  Detail & Related papers  (2022-11-22T20:39:18Z)
• Enhanced Modality Transition for Image Captioning [51.72997126838352]
  We build a Modality Transition Module (MTM) to transfer visual features into semantic representations before forwarding them to the language model. During the training phase, the modality transition network is optimised by the proposed modality loss. Experiments have been conducted on the MS-COCO dataset demonstrating the effectiveness of the proposed framework.
  arXiv  Detail & Related papers  (2021-02-23T07:20:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.