Uncovering Regional Defaults from Photorealistic Forests in Text-to-Image Generation with DALL-E 2

• URL: http://arxiv.org/abs/2410.17255v1
• Date: Thu, 03 Oct 2024 19:07:12 GMT
• Title: Uncovering Regional Defaults from Photorealistic Forests in Text-to-Image Generation with DALL-E 2
• Abstract summary: Regional defaults describe the emerging phenomenon that text-to-image (T2I) foundation models are prone to over-proportionally depicting certain geographic regions. In this work, we introduce a scalable evaluation for uncovering such regional defaults. Our findings underscore a need for further investigation into the geography of T2I generation and other forms of generative AI.
• Score: 5.534517268996598
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Regional defaults describe the emerging phenomenon that text-to-image (T2I) foundation models used in generative AI are prone to over-proportionally depicting certain geographic regions to the exclusion of others. In this work, we introduce a scalable evaluation for uncovering such regional defaults. The evaluation consists of region hierarchy--based image generation and cross-level similarity comparisons. We carry out an experiment by prompting DALL-E 2, a state-of-the-art T2I generation model capable of generating photorealistic images, to depict a forest. We select forest as an object class that displays regional variation and can be characterized using spatial statistics. For a region in the hierarchy, our experiment reveals the regional defaults implicit in DALL-E 2, along with their scale-dependent nature and spatial relationships. In addition, we discover that the implicit defaults do not necessarily correspond to the most widely forested regions in reality. Our findings underscore a need for further investigation into the geography of T2I generation and other forms of generative AI.

Related papers

• RAGSR: Regional Attention Guided Diffusion for Image Super-Resolution [38.794214985205045]
  We propose a novel method to generate clear and accurate regional details in super-resolution images.<n>The method explicitly extracts localized fine-grained information and encodes it through a novel regional attention mechanism.<n> Experimental results on benchmark datasets demonstrate that our approach exhibits superior performance in generating perceptually authentic visual details.
  arXiv  Detail & Related papers  (2025-08-22T07:28:34Z)
• AI's Blind Spots: Geographic Knowledge and Diversity Deficit in Generated Urban Scenario [4.315451628809687]
  We generated 150 synthetic images for each state in the USA and related capitals using two state-of-the-art models for image generation.<n>We found that while these models have implicitly learned aspects of USA geography, if we prompt the models to generate an image for "United States" instead of specific cities or states, the models exhibit a strong representative bias toward metropolis-like areas.
  arXiv  Detail & Related papers  (2025-06-20T10:43:22Z)
• SounDiT: Geo-Contextual Soundscape-to-Landscape Generation [28.099729084181092]
  We present a novel problem-Geo-Contextual Soundscape-to-Landscape (GeoS2L) generation.<n>GeoS2L aims to synthesize geographically realistic landscape images from environmental soundscapes.
  arXiv  Detail & Related papers  (2025-05-19T05:47:13Z)
• Visual Text Generation in the Wild [67.37458807253064]
  We propose a visual text generator (termed SceneVTG) which can produce high-quality text images in the wild. The proposed SceneVTG significantly outperforms traditional rendering-based methods and recent diffusion-based methods in terms of fidelity and reasonability. The generated images provide superior utility for tasks involving text detection and text recognition.
  arXiv  Detail & Related papers  (2024-07-19T09:08:20Z)
• RTGen: Generating Region-Text Pairs for Open-Vocabulary Object Detection [20.630629383286262]
  Open-vocabulary object detection requires solid modeling of the region-semantic relationship. We propose RTGen to generate scalable open-vocabulary region-text pairs.
  arXiv  Detail & Related papers  (2024-05-30T09:03:23Z)
• R&B: Region and Boundary Aware Zero-shot Grounded Text-to-image Generation [74.5598315066249]
  We probe into zero-shot grounded T2I generation with diffusion models. We propose a Region and Boundary (R&B) aware cross-attention guidance approach.
  arXiv  Detail & Related papers  (2023-10-13T05:48:42Z)
• LAW-Diffusion: Complex Scene Generation by Diffusion with Layouts [107.11267074981905]
  We propose a semantically controllable layout-AWare diffusion model, termed LAW-Diffusion. We show that LAW-Diffusion yields the state-of-the-art generative performance, especially with coherent object relations.
  arXiv  Detail & Related papers  (2023-08-13T08:06:18Z)
• Social Biases through the Text-to-Image Generation Lens [9.137275391251517]
  Text-to-Image (T2I) generation is enabling new applications that support creators, designers, and general end users of productivity software. We take a multi-dimensional approach to studying and quantifying common social biases as reflected in the generated images. We present findings for two popular T2I models: DALLE-v2 and Stable Diffusion.
  arXiv  Detail & Related papers  (2023-03-30T05:29:13Z)
• GIVL: Improving Geographical Inclusivity of Vision-Language Models with Pre-Training Methods [62.076647211744564]
  We propose GIVL, a Geographically Inclusive Vision-and-Language Pre-trained model. There are two attributes of geo-diverse visual concepts which can help to learn geo-diverse knowledge: 1) concepts under similar categories have unique knowledge and visual characteristics, 2) concepts with similar visual features may fall in completely different categories. Compared with similar-size models pre-trained with similar scale of data, GIVL achieves state-of-the-art (SOTA) and more balanced performance on geo-diverse V&L tasks.
  arXiv  Detail & Related papers  (2023-01-05T03:43:45Z)
• RegionCLIP: Region-based Language-Image Pretraining [94.29924084715316]
  Contrastive language-image pretraining (CLIP) using image-text pairs has achieved impressive results on image classification. We propose a new method called RegionCLIP that significantly extends CLIP to learn region-level visual representations. Our method significantly outperforms the state of the art by 3.8 AP50 and 2.2 AP for novel categories on COCO and LVIS datasets.
  arXiv  Detail & Related papers  (2021-12-16T18:39:36Z)
• Formatting the Landscape: Spatial conditional GAN for varying population in satellite imagery [0.0]
  Changes to the geographic distribution of population will have dramatic impacts on land use and land cover. We explore a generative model framework for generating satellite imagery conditional on gridded population distributions.
  arXiv  Detail & Related papers  (2020-12-08T13:31:49Z)
• Local Class-Specific and Global Image-Level Generative Adversarial Networks for Semantic-Guided Scene Generation [135.4660201856059]
  We consider learning the scene generation in a local context, and design a local class-specific generative network with semantic maps as a guidance. To learn more discrimi class-specific feature representations for the local generation, a novel classification module is also proposed. Experiments on two scene image generation tasks show superior generation performance of the proposed model.
  arXiv  Detail & Related papers  (2019-12-27T16:14:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.