Seedream 2.0: A Native Chinese-English Bilingual Image Generation Foundation Model

• URL: http://arxiv.org/abs/2503.07703v1
• Date: Mon, 10 Mar 2025 17:58:33 GMT
• Title: Seedream 2.0: A Native Chinese-English Bilingual Image Generation Foundation Model
• Authors: Lixue Gong, Xiaoxia Hou, Fanshi Li, Liang Li, Xiaochen Lian, Fei Liu, Liyang Liu, Wei Liu, Wei Lu, Yichun Shi, Shiqi Sun, Yu Tian, Zhi Tian, Peng Wang, Xun Wang, Ye Wang, Guofeng Wu, Jie Wu, Xin Xia, Xuefeng Xiao, Linjie Yang, Zhonghua Zhai, Xinyu Zhang, Qi Zhang, Yuwei Zhang, Shijia Zhao, Jianchao Yang, Weilin Huang,
• Abstract summary: Seedream 2.0 is a native Chinese-English bilingual image generation foundation model.<n>It adeptly manages text prompt in both Chinese and English, supporting bilingual image generation and text rendering.<n>It is integrated with a self-developed bilingual large language model as a text encoder, allowing it to learn native knowledge directly from massive data.
• Score: 69.09404597939744
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Rapid advancement of diffusion models has catalyzed remarkable progress in the field of image generation. However, prevalent models such as Flux, SD3.5 and Midjourney, still grapple with issues like model bias, limited text rendering capabilities, and insufficient understanding of Chinese cultural nuances. To address these limitations, we present Seedream 2.0, a native Chinese-English bilingual image generation foundation model that excels across diverse dimensions, which adeptly manages text prompt in both Chinese and English, supporting bilingual image generation and text rendering. We develop a powerful data system that facilitates knowledge integration, and a caption system that balances the accuracy and richness for image description. Particularly, Seedream is integrated with a self-developed bilingual large language model as a text encoder, allowing it to learn native knowledge directly from massive data. This enable it to generate high-fidelity images with accurate cultural nuances and aesthetic expressions described in either Chinese or English. Beside, Glyph-Aligned ByT5 is applied for flexible character-level text rendering, while a Scaled ROPE generalizes well to untrained resolutions. Multi-phase post-training optimizations, including SFT and RLHF iterations, further improve the overall capability. Through extensive experimentation, we demonstrate that Seedream 2.0 achieves state-of-the-art performance across multiple aspects, including prompt-following, aesthetics, text rendering, and structural correctness. Furthermore, Seedream 2.0 has been optimized through multiple RLHF iterations to closely align its output with human preferences, as revealed by its outstanding ELO score. In addition, it can be readily adapted to an instruction-based image editing model, such as SeedEdit, with strong editing capability that balances instruction-following and image consistency.

Related papers

• Seedream 3.0 Technical Report [62.85849652170507]
  Seedream 3.0 is a high-performance Chinese-English bilingual image generation foundation model. We develop several technical improvements to address existing challenges in Seedream 2.0. Seedream 3.0 provides native high-resolution output (up to 2K) allowing it to generate images with high visual quality.
  arXiv  Detail & Related papers  (2025-04-15T16:19:07Z)
• Visual Lexicon: Rich Image Features in Language Space [99.94214846451347]
  ViLex simultaneously captures rich semantic content and fine visual details.<n>ViLex generates tokens optimized for reconstructing input images using a frozen text-to-image (T2I) diffusion model.<n>As an image embedding in the language space, ViLex tokens leverage the compositionality of natural languages.
  arXiv  Detail & Related papers  (2024-12-09T18:57:24Z)
• ARTIST: Improving the Generation of Text-rich Images with Disentangled Diffusion Models and Large Language Models [52.23899502520261]
  We introduce a novel framework named, ARTIST, which incorporates a dedicated textual diffusion model to focus on the learning of text structures specifically.<n>We finetune a visual diffusion model, enabling it to assimilate textual structure information from the pretrained textual model.<n>This disentangled architecture design and training strategy significantly enhance the text rendering ability of the diffusion models for text-rich image generation.
  arXiv  Detail & Related papers  (2024-06-17T19:31:24Z)
• AnyTrans: Translate AnyText in the Image with Large Scale Models [88.5887934499388]
  This paper introduces AnyTrans, an all-encompassing framework for the task-Translate AnyText in the Image (TATI) Our framework incorporates contextual cues from both textual and visual elements during translation. We have meticulously compiled a test dataset called MTIT6, which consists of multilingual text image translation data from six language pairs.
  arXiv  Detail & Related papers  (2024-06-17T11:37:48Z)
• Reason out Your Layout: Evoking the Layout Master from Large Language Models for Text-to-Image Synthesis [47.27044390204868]
  We introduce a novel approach to improving T2I diffusion models using Large Language Models (LLMs) as layout generators. Our experiments demonstrate significant improvements in image quality and layout accuracy.
  arXiv  Detail & Related papers  (2023-11-28T14:51:13Z)
• Paragraph-to-Image Generation with Information-Enriched Diffusion Model [67.9265336953134]
  ParaDiffusion is an information-enriched diffusion model for paragraph-to-image generation task. It delves into the transference of the extensive semantic comprehension capabilities of large language models to the task of image generation. The code and dataset will be released to foster community research on long-text alignment.
  arXiv  Detail & Related papers  (2023-11-24T05:17:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.