Creatively Upscaling Images with Global-Regional Priors

• URL: http://arxiv.org/abs/2505.16976v1
• Date: Thu, 22 May 2025 17:51:50 GMT
• Title: Creatively Upscaling Images with Global-Regional Priors
• Abstract summary: C-Upscale is a new recipe of tuning-free image upscaling.<n>It pivots on global-regional priors derived from given global prompt and estimated regional prompts.<n>It generates ultra-high-resolution images with higher visual fidelity and more creative regional details.
• Score: 98.24171965992916
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Contemporary diffusion models show remarkable capability in text-to-image generation, while still being limited to restricted resolutions (e.g., 1,024 X 1,024). Recent advances enable tuning-free higher-resolution image generation by recycling pre-trained diffusion models and extending them via regional denoising or dilated sampling/convolutions. However, these models struggle to simultaneously preserve global semantic structure and produce creative regional details in higher-resolution images. To address this, we present C-Upscale, a new recipe of tuning-free image upscaling that pivots on global-regional priors derived from given global prompt and estimated regional prompts via Multimodal LLM. Technically, the low-frequency component of low-resolution image is recognized as global structure prior to encourage global semantic consistency in high-resolution generation. Next, we perform regional attention control to screen cross-attention between global prompt and each region during regional denoising, leading to regional attention prior that alleviates object repetition issue. The estimated regional prompts containing rich descriptive details further act as regional semantic prior to fuel the creativity of regional detail generation. Both quantitative and qualitative evaluations demonstrate that our C-Upscale manages to generate ultra-high-resolution images (e.g., 4,096 X 4,096 and 8,192 X 8,192) with higher visual fidelity and more creative regional details.

Related papers

• DreamSR: Towards Ultra-High-Resolution Image Super-Resolution via a Receptive-Field Enhanced Diffusion Transformer [18.545061362433362]
  We present DreamSR, a novel SR model that suppresses local over-generation and improves fine-detail synthesis.<n>We also design a comprehensive training strategy with stage-specific data processing pipelines and a Receptive-Field Enhancement strategy.
  arXiv  Detail & Related papers  (2026-05-15T07:08:01Z)
• AINet: Anchor Instances Learning for Regional Heterogeneity in Whole Slide Image [61.54860340942449]
  We introduce a novel concept of anchor instance (AI), a compact subset of instances that are representative within their regions (local) and discriminative at the bag (global) level.<n>These AIs act as semantic references to guide interactions across regions, correcting non-discriminative patterns while preserving regional diversity.<n>We develop a concise yet effective framework, AINet, which employs a simple predictor and surpasses state-of-the-art methods with substantially fewer FLOPs and parameters.
  arXiv  Detail & Related papers  (2026-02-21T09:36:27Z)
• HarmoCLIP: Harmonizing Global and Regional Representations in Contrastive Vision-Language Models [63.87966115136411]
  HarmoCLIP is a novel framework designed to harmonize global and region representations within Contrastive Language-Image Pre-training.<n>To strengthen the representation capability at the local level, our method introduces a novel Region-Language Alignment supervision strategy.
  arXiv  Detail & Related papers  (2025-11-27T16:24:53Z)
• Scale-DiT: Ultra-High-Resolution Image Generation with Hierarchical Local Attention [50.391914489898774]
  Scale-DiT is a new diffusion framework that introduces hierarchical local attention with low-resolution global guidance.<n>A lightweight LoRA adaptation bridges global and local pathways during denoising, ensuring consistency across structure and detail.<n>Experiments demonstrate that Scale-DiT achieves more than $2times$ faster inference and lower memory usage.
  arXiv  Detail & Related papers  (2025-10-18T03:15:26Z)
• RAGSR: Regional Attention Guided Diffusion for Image Super-Resolution [38.794214985205045]
  We propose a novel method to generate clear and accurate regional details in super-resolution images.<n>The method explicitly extracts localized fine-grained information and encodes it through a novel regional attention mechanism.<n> Experimental results on benchmark datasets demonstrate that our approach exhibits superior performance in generating perceptually authentic visual details.
  arXiv  Detail & Related papers  (2025-08-22T07:28:34Z)
• Segmenting and Understanding: Region-aware Semantic Attention for Fine-grained Image Quality Assessment with Large Language Models [15.08436609204683]
  No-reference image quality assessment (NR-IQA) aims to simulate the process of perceiving image quality aligned with human perception.<n>We propose a fine-grained image quality assessment model, named RSFIQA, which integrates region-level distortion information to perceive multi-dimensional quality discrepancies.
  arXiv  Detail & Related papers  (2025-08-11T10:03:00Z)
• From Missing Pieces to Masterpieces: Image Completion with Context-Adaptive Diffusion [98.31811240195324]
  ConFill is a novel framework that reduces discrepancies between generated and original images at each diffusion step.<n>It outperforms current methods, setting a new benchmark in image completion.
  arXiv  Detail & Related papers  (2025-04-19T13:40:46Z)
• Can Location Embeddings Enhance Super-Resolution of Satellite Imagery? [2.3020018305241337]
  Publicly available satellite imagery, such as Sentinel- 2, often lacks the spatial resolution required for accurate analysis of remote sensing tasks.<n>We propose a novel super-resolution framework that enhances generalization by incorporating geographic context through location embeddings.<n>We demonstrate the effectiveness of our method on the building segmentation task, showing significant improvements over state-of-the-art methods.
  arXiv  Detail & Related papers  (2025-01-27T08:16:54Z)
• Region-Aware Text-to-Image Generation via Hard Binding and Soft Refinement [40.94329069897935]
  We present RAG, a Regional-Aware text-to-image Generation method conditioned on regional descriptions for precise layout composition. RAG achieves superior performance over attribute binding and object relationship than previous tuning-free methods.
  arXiv  Detail & Related papers  (2024-11-10T18:45:41Z)
• HiPrompt: Tuning-free Higher-Resolution Generation with Hierarchical MLLM Prompts [77.62320553269615]
  HiPrompt is a tuning-free solution for higher-resolution image generation. hierarchical prompts offer both global and local guidance. generated images maintain coherent local and global semantics, structures, and textures with high definition.
  arXiv  Detail & Related papers  (2024-09-04T17:58:08Z)
• Zero-shot Text-guided Infinite Image Synthesis with LLM guidance [2.531998650341267]
  There is a lack of text-image paired datasets with high-resolution and contextual diversity.<n>Expanding images based on text requires global coherence and rich local context understanding.<n>We propose a novel approach utilizing Large Language Models (LLMs) for both global coherence and local context understanding.
  arXiv  Detail & Related papers  (2024-07-17T15:10:01Z)
• Coherent and Multi-modality Image Inpainting via Latent Space Optimization [61.99406669027195]
  PILOT (intextbfPainting vtextbfIa textbfLatent textbfOptextbfTimization) is an optimization approach grounded on a novel textitsemantic centralization and textitbackground preservation loss. Our method searches latent spaces capable of generating inpainted regions that exhibit high fidelity to user-provided prompts while maintaining coherence with the background.
  arXiv  Detail & Related papers  (2024-07-10T19:58:04Z)
• RegionGPT: Towards Region Understanding Vision Language Model [88.42271128373191]
  RegionGPT (short as RGPT) is a novel framework designed for complex region-level captioning and understanding. We develop an automated region caption data generation pipeline, enriching the training set with detailed region-level captions. We demonstrate that a universal RGPT model can be effectively applied and significantly enhancing performance across a range of region-level tasks.
  arXiv  Detail & Related papers  (2024-03-04T18:58:08Z)
• Efficient and Explicit Modelling of Image Hierarchies for Image Restoration [120.35246456398738]
  We propose a mechanism to efficiently and explicitly model image hierarchies in the global, regional, and local range for image restoration. Inspired by that, we propose the anchored stripe self-attention which achieves a good balance between the space and time complexity of self-attention. Then we propose a new network architecture dubbed GRL to explicitly model image hierarchies in the Global, Regional, and Local range.
  arXiv  Detail & Related papers  (2023-03-01T18:59:29Z)
• GIU-GANs: Global Information Utilization for Generative Adversarial Networks [3.3945834638760948]
  In this paper, we propose a new GANs called Involution Generative Adversarial Networks (GIU-GANs) GIU-GANs leverages a brand new module called the Global Information Utilization (GIU) module, which integrates Squeeze-and-Excitation Networks (SENet) and involution. Batch Normalization(BN) inevitably ignores the representation differences among noise sampled by the generator, and thus degrades the generated image quality.
  arXiv  Detail & Related papers  (2022-01-25T17:17:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.