Revisiting the Perception-Distortion Trade-off with Spatial-Semantic Guided Super-Resolution

• URL: http://arxiv.org/abs/2603.14112v1
• Date: Sat, 14 Mar 2026 20:41:27 GMT
• Title: Revisiting the Perception-Distortion Trade-off with Spatial-Semantic Guided Super-Resolution
• Abstract summary: SpaSemSR is a spatial-semantic guided diffusion framework with two complementary guidances.<n>First, spatial-grounded textual guidance integrates object-level spatial cues with semantic prompts, aligning textual and visual structures to reduce distortion.<n>Second, semantic-enhanced visual guidance with a multi-encoder design and semantic degradation constraints unifies multimodal semantic priors, improving perceptual realism under severe degradations.
• Score: 25.911279124430944
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Image super-resolution (SR) aims to reconstruct high resolution images with both high perceptual quality and low distortion, but is fundamentally limited by the perception-distortion trade-off. GAN-based SR methods reduce distortion but still struggle with realistic fine-grained textures, whereas diffusion-based approaches synthesize rich details but often deviate from the input, hallucinating structures and degrading fidelity. This tension raises a key challenge: how to exploit the powerful generative priors of diffusion models without sacrificing fidelity. To address this, we propose SpaSemSR, a spatial-semantic guided diffusion framework with two complementary guidances. First, spatial-grounded textual guidance integrates object-level spatial cues with semantic prompts, aligning textual and visual structures to reduce distortion. Second, semantic-enhanced visual guidance with a multi-encoder design and semantic degradation constraints unifies multimodal semantic priors, improving perceptual realism under severe degradations. These complementary guidances are adaptively fused into the diffusion process via spatial-semantic attention, suppressing distortion and hallucination while retaining the strengths of diffusion models. Extensive experiments on multiple benchmarks show that SpaSemSR achieves a superior perception-distortion balance, producing both realistic and faithful restorations.

Related papers

• Rarity-Aware Discrete Diffusion with Spatially Consistent Decoding for Photo-Realistic Image Super-Resolution [80.28905986756685]
  DiMOO-SR is a rarity-aware multimodal discrete diffusion framework for photo-realistic image Super-Resolution.<n>Experiments on widely used real-world SR benchmarks demonstrate that DiMOO-SR achieves competitive perceptual quality with only a few parallel decoding steps.
  arXiv  Detail & Related papers  (2026-07-20T07:01:01Z)
• Beyond Unfolding: 60x Faster One-Stage Unmixing for Closely-Spaced Infrared Small Targets [43.71958109614362]
  Closely-Spaced Infrared Small Targets (CSIST) typically exhibit energy overlap, manifesting as indistinguishable blobs in infrared images.<n>This ambiguity invalidates the one-to-one mapping assumption of traditional detection.<n>We propose a one-stage lightweight paradigm which demonstrates that deep unfolding is not necessary.
  arXiv  Detail & Related papers  (2026-07-17T14:42:51Z)
• Better with Less: Tackling Heterogeneous Multi-Modal Image Joint Pretraining via Conditioned and Degraded Masked Autoencoder [34.73963627819185]
  High-resolution (HR) joint optical and synthetic aperture radar (SAR) pretraining seeks modality synergy to mutually enhance single-source representations.<n>We propose CoDe-MAE, pioneering a textitbetter synergy with less alignment philosophy.<n>CoDe-MAE successfully prevents representation degradation and establishes new state-of-the-art performance across diverse single- and bi-modal downstream tasks.
  arXiv  Detail & Related papers  (2026-04-18T10:23:00Z)
• Customizing Large Vision Model-Guided Low-Rank Approximation for Ground-Roll Denoise [4.536244262111516]
  Ground-roll is a dominant source of coherent noise in land and vertical seismic profiling (VSP) data.<n>We propose a training-free framework that reformulates ground-roll attenuation as a semantic-guided signal separation problem.
  arXiv  Detail & Related papers  (2026-04-01T14:59:48Z)
• FUMO: Prior-Modulated Diffusion for Single Image Reflection Removal [51.98991734660274]
  This paper presents a diffusion model with prior modulation framework (FUMO) that introduces explicit guidance signals to improve spatial controllability and structural faithfulness.<n>Experiments conducted on both standard benchmarks and challenging images in the wild demonstrate competitive quantitative results and consistently improved perceptual quality.
  arXiv  Detail & Related papers  (2026-03-19T15:33:11Z)
• Disentangled Textual Priors for Diffusion-based Image Super-Resolution [41.71306518338786]
  Image Super-Resolution aims to reconstruct high-resolution images from degraded low-resolution inputs.<n>Existing approaches often rely on entangled or coarse-grained priors that mix global layout with local details.<n>We propose DTPSR, a novel diffusion-based SR framework that introduces disentangled textual priors along two complementary dimensions.
  arXiv  Detail & Related papers  (2026-03-08T03:02:55Z)
• Rectifying Latent Space for Generative Single-Image Reflection Removal [16.341477336909765]
  Single-image removal is a highly ill-posed problem, where existing methods struggle to reason about the composition of corrupted regions.<n>This work reframes an editing-purpose latent diffusion model to effectively perceive and process highly ambiguous, layered image inputs.
  arXiv  Detail & Related papers  (2025-12-06T09:16:14Z)
• OmniScaleSR: Unleashing Scale-Controlled Diffusion Prior for Faithful and Realistic Arbitrary-Scale Image Super-Resolution [25.29121925794166]
  Arbitrary-scale super-resolution (ASSR) overcomes the limitation of traditional super-resolution (SR) methods that operate only at fixed scales (e.g., 4x)<n>Recent diffusion-based realistic image super-resolution (Real-ISR) models leverage powerful pre-trained diffusion priors and show impressive results at the 4x setting.<n>We propose OmniScaleSR, a diffusion-based realistic arbitrary-scale SR framework designed to achieve both high fidelity and high realism.
  arXiv  Detail & Related papers  (2025-12-04T11:50:17Z)
• Edit2Perceive: Image Editing Diffusion Models Are Strong Dense Perceivers [55.15722080205737]
  Edit2Perceive is a unified diffusion framework that adapts editing models for depth, normal, and matting.<n>Our single-step deterministic inference yields up to faster runtime while training on relatively small datasets.
  arXiv  Detail & Related papers  (2025-11-24T01:13:51Z)
• SRSR: Enhancing Semantic Accuracy in Real-World Image Super-Resolution with Spatially Re-Focused Text-Conditioning [59.013863248600046]
  We propose a spatially re-focused super-resolution framework that refines text conditioning at inference time.<n>Second, we introduce a Spatially Targeted-Free Guidance mechanism that selectively bypasses text influences on ungrounded pixels to prevent hallucinations.
  arXiv  Detail & Related papers  (2025-10-26T05:03:55Z)
• Boosting Fidelity for Pre-Trained-Diffusion-Based Low-Light Image Enhancement via Condition Refinement [63.54516423266521]
  Pre-Trained Diffusion-Based (PTDB) methods often sacrifice content fidelity to attain higher perceptual realism.<n>We propose a novel optimization strategy for conditioning in pre-trained diffusion models, enhancing fidelity while preserving realism and aesthetics.<n>Our approach is plug-and-play, seamlessly integrating into existing diffusion networks to provide more effective control.
  arXiv  Detail & Related papers  (2025-10-20T02:40:06Z)
• One-Step Diffusion-based Real-World Image Super-Resolution with Visual Perception Distillation [53.24542646616045]
  We propose VPD-SR, a novel visual perception diffusion distillation framework specifically designed for image super-resolution (SR) generation.<n>VPD-SR consists of two components: Explicit Semantic-aware Supervision (ESS) and High-frequency Perception (HFP) loss.<n>The proposed VPD-SR achieves superior performance compared to both previous state-of-the-art methods and the teacher model with just one-step sampling.
  arXiv  Detail & Related papers  (2025-06-03T08:28:13Z)
• Multimodal LLM-Guided Semantic Correction in Text-to-Image Diffusion [52.315729095824906]
  MLLM Semantic-Corrected Ping-Pong-Ahead Diffusion (PPAD) is a novel framework that introduces a Multimodal Large Language Model (MLLM) as a semantic observer during inference.<n>It performs real-time analysis on intermediate generations, identifies latent semantic inconsistencies, and translates feedback into controllable signals that actively guide the remaining denoising steps.<n>Extensive experiments demonstrate PPAD's significant improvements.
  arXiv  Detail & Related papers  (2025-05-26T14:42:35Z)
• Enhanced Semantic Extraction and Guidance for UGC Image Super Resolution [18.058473238611725]
  We propose a novel approach to image super-resolution by integrating semantic guidance into a diffusion framework.<n>Our method addresses the inconsistency between degradations in wild and synthetic datasets.<n>Our model won second place in the CVIRE 2025 Short-form Image Super-Resolution Challenge.
  arXiv  Detail & Related papers  (2025-04-14T05:26:24Z)
• Denoising as Adaptation: Noise-Space Domain Adaptation for Image Restoration [64.84134880709625]
  We show that it is possible to perform domain adaptation via the noise space using diffusion models.<n>In particular, by leveraging the unique property of how auxiliary conditional inputs influence the multi-step denoising process, we derive a meaningful diffusion loss.<n>We present crucial strategies such as channel-shuffling layer and residual-swapping contrastive learning in the diffusion model.
  arXiv  Detail & Related papers  (2024-06-26T17:40:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.