One-Step Diffusion for Detail-Rich and Temporally Consistent Video Super-Resolution

• URL: http://arxiv.org/abs/2506.15591v2
• Date: Fri, 20 Jun 2025 15:05:29 GMT
• Title: One-Step Diffusion for Detail-Rich and Temporally Consistent Video Super-Resolution
• Authors: Yujing Sun, Lingchen Sun, Shuaizheng Liu, Rongyuan Wu, Zhengqiang Zhang, Lei Zhang,
• Abstract summary: We propose a Dual LoRA Learning (DLoRAL) paradigm to train an effective SD-based one-step diffusion model.<n> Experiments show that DLoRAL achieves strong performance in both accuracy and speed.
• Score: 9.03810927740921
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: It is a challenging problem to reproduce rich spatial details while maintaining temporal consistency in real-world video super-resolution (Real-VSR), especially when we leverage pre-trained generative models such as stable diffusion (SD) for realistic details synthesis. Existing SD-based Real-VSR methods often compromise spatial details for temporal coherence, resulting in suboptimal visual quality. We argue that the key lies in how to effectively extract the degradation-robust temporal consistency priors from the low-quality (LQ) input video and enhance the video details while maintaining the extracted consistency priors. To achieve this, we propose a Dual LoRA Learning (DLoRAL) paradigm to train an effective SD-based one-step diffusion model, achieving realistic frame details and temporal consistency simultaneously. Specifically, we introduce a Cross-Frame Retrieval (CFR) module to aggregate complementary information across frames, and train a Consistency-LoRA (C-LoRA) to learn robust temporal representations from degraded inputs. After consistency learning, we fix the CFR and C-LoRA modules and train a Detail-LoRA (D-LoRA) to enhance spatial details while aligning with the temporal space defined by C-LoRA to keep temporal coherence. The two phases alternate iteratively for optimization, collaboratively delivering consistent and detail-rich outputs. During inference, the two LoRA branches are merged into the SD model, allowing efficient and high-quality video restoration in a single diffusion step. Experiments show that DLoRAL achieves strong performance in both accuracy and speed. Code and models are available at https://github.com/yjsunnn/DLoRAL.

Related papers

• UltraVSR: Achieving Ultra-Realistic Video Super-Resolution with Efficient One-Step Diffusion Space [46.43409853027655]
  UltraVSR is a framework that enables ultra-realistic and temporal-coherent VSR.<n>DRS transforms denoising process into a single-step reconstruction from low-resolution to high-resolution videos.<n> RTS module is integrated into a pretrained text-to-image diffusion model.
  arXiv  Detail & Related papers  (2025-05-26T13:19:27Z)
• DC-VSR: Spatially and Temporally Consistent Video Super-Resolution with Video Diffusion Prior [13.324336907242195]
  Video-resolution (VSR) aims to reconstruct a high-resolution (HR) video from a low-resolution (LR) counterpart.<n>DC-VSR produces spatially and temporally consistent VSR results with realistic textures.<n> experiments demonstrate that DC-VSR achieves spatially and temporally consistent high-quality VSR results, outperforming previous approaches.
  arXiv  Detail & Related papers  (2025-02-05T10:15:00Z)
• Collaborative Feedback Discriminative Propagation for Video Super-Resolution [66.61201445650323]
  Key success of video super-resolution (VSR) methods stems mainly from exploring spatial and temporal information. Inaccurate alignment usually leads to aligned features with significant artifacts. propagation modules only propagate the same timestep features forward or backward.
  arXiv  Detail & Related papers  (2024-04-06T22:08:20Z)
• Upscale-A-Video: Temporal-Consistent Diffusion Model for Real-World Video Super-Resolution [65.91317390645163]
  Upscale-A-Video is a text-guided latent diffusion framework for video upscaling. It ensures temporal coherence through two key mechanisms: locally, it integrates temporal layers into U-Net and VAE-Decoder, maintaining consistency within short sequences. It also offers greater flexibility by allowing text prompts to guide texture creation and adjustable noise levels to balance restoration and generation.
  arXiv  Detail & Related papers  (2023-12-11T18:54:52Z)
• Motion-Guided Latent Diffusion for Temporally Consistent Real-world Video Super-resolution [15.197746480157651]
  We propose an effective real-world VSR algorithm by leveraging the strength of pre-trained latent diffusion models. We exploit the temporal dynamics in LR videos to guide the diffusion process by optimizing the latent sampling path with a motion-guided loss. The proposed motion-guided latent diffusion based VSR algorithm achieves significantly better perceptual quality than state-of-the-arts on real-world VSR benchmark datasets.
  arXiv  Detail & Related papers  (2023-12-01T14:40:07Z)
• Local-Global Temporal Difference Learning for Satellite Video Super-Resolution [53.03380679343968]
  We propose to exploit the well-defined temporal difference for efficient and effective temporal compensation.<n>To fully utilize the local and global temporal information within frames, we systematically modeled the short-term and long-term temporal discrepancies.<n> Rigorous objective and subjective evaluations conducted across five mainstream video satellites demonstrate that our method performs favorably against state-of-the-art approaches.
  arXiv  Detail & Related papers  (2023-04-10T07:04:40Z)
• You Only Align Once: Bidirectional Interaction for Spatial-Temporal Video Super-Resolution [14.624610700550754]
  We propose an efficient recurrent network with bidirectional interaction for ST-VSR. It first performs backward inference from future to past, and then follows forward inference to super-resolve intermediate frames. Our method outperforms state-of-the-art methods in efficiency, and reduces calculation cost by about 22%.
  arXiv  Detail & Related papers  (2022-07-13T17:01:16Z)
• Look Back and Forth: Video Super-Resolution with Explicit Temporal Difference Modeling [105.69197687940505]
  We propose to explore the role of explicit temporal difference modeling in both LR and HR space. To further enhance the super-resolution result, not only spatial residual features are extracted, but the difference between consecutive frames in high-frequency domain is also computed.
  arXiv  Detail & Related papers  (2022-04-14T17:07:33Z)
• STDAN: Deformable Attention Network for Space-Time Video Super-Resolution [39.18399652834573]
  We propose a deformable attention network called STDAN for STVSR. First, we devise a long-short term feature (LSTFI) module, which is capable of abundant content from more neighboring input frames. Second, we put forward a spatial-temporal deformable feature aggregation (STDFA) module, in which spatial and temporal contexts are adaptively captured and aggregated.
  arXiv  Detail & Related papers  (2022-03-14T03:40:35Z)
• Zooming SlowMo: An Efficient One-Stage Framework for Space-Time Video Super-Resolution [100.11355888909102]
  Space-time video super-resolution aims at generating a high-resolution (HR) slow-motion video from a low-resolution (LR) and low frame rate (LFR) video sequence. We present a one-stage space-time video super-resolution framework, which can directly reconstruct an HR slow-motion video sequence from an input LR and LFR video.
  arXiv  Detail & Related papers  (2021-04-15T17:59:23Z)
• Zooming Slow-Mo: Fast and Accurate One-Stage Space-Time Video Super-Resolution [95.26202278535543]
  A simple solution is to split it into two sub-tasks: video frame (VFI) and video super-resolution (VSR) temporalsynthesis and spatial super-resolution are intra-related in this task. We propose a one-stage space-time video super-resolution framework, which directly synthesizes an HR slow-motion video from an LFR, LR video.
  arXiv  Detail & Related papers  (2020-02-26T16:59:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.