Frames2Residual: Spatiotemporal Decoupling for Self-Supervised Video Denoising

• URL: http://arxiv.org/abs/2603.10417v1
• Date: Wed, 11 Mar 2026 05:04:28 GMT
• Title: Frames2Residual: Spatiotemporal Decoupling for Self-Supervised Video Denoising
• Abstract summary: Self-supervised video denoising methods often struggle to integrate inter-frame temporal consistency with intra-frame spatial specificity.<n>Existing Video Blind-Spot- Networks (BSNs) require noise independence by masking the center pixel, this constraint prevents the use of texture for spatial evidence recovery.<n>We propose loss2Residuals (F2R), a framework that divides self-supervised training into two distinct stages: blind temporal consistency modeling and non-blind spatial texture recovery.
• Score: 28.397068445063724
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Self-supervised video denoising methods typically extend image-based frameworks into the temporal dimension, yet they often struggle to integrate inter-frame temporal consistency with intra-frame spatial specificity. Existing Video Blind-Spot Networks (BSNs) require noise independence by masking the center pixel, this constraint prevents the use of spatial evidence for texture recovery, thereby severing spatiotemporal correlations and causing texture loss. To address this, we propose Frames2Residual (F2R), a spatiotemporal decoupling framework that explicitly divides self-supervised training into two distinct stages: blind temporal consistency modeling and non-blind spatial texture recovery. In Stage 1, a blind temporal estimator learns inter-frame consistency using a frame-wise blind strategy, producing a temporally consistent anchor. In Stage 2, a non-blind spatial refiner leverages this anchor to safely reintroduce the center frame and recover intra-frame high-frequency spatial residuals while preserving temporal stability. Extensive experiments demonstrate that our decoupling strategy allows F2R to outperform existing self-supervised methods on both sRGB and raw video benchmarks.

Related papers

• Distilling Temporal Coherence into 2D Networks for Transrectal Ultrasound Prostate Video Segmentation [2.65131397683555]
  Real-time video segmentation of the prostate in Transrectal Ultrasound (TRUS) is essential for image-guided interventions.<n>We present a Temporally Consistent Learning Framework that distills temporal coherence into a 2D network during training.<n>Our design is driven by a key clinical observation: the prostate exhibits geometric stability, whereas the surrounding acoustic environment fluctuates due to physiological motion and transducer pressure.
  arXiv  Detail & Related papers  (2026-06-30T06:27:02Z)
• RhymeFlow: Training-Free Acceleration for Video Generation with Asynchronous Denoising Flow Scheduling [51.279397568734424]
  Video generation models based on Diffusion Transformers (DiTs) have achieved remarkable performance in video synthesis.<n>DiTs suffer from high inference latency and computational costs due to the quadratic complexity of 3D attention.<n>We introduce bftextRhymeFlow, a training-free framework that decouples the denoising trajectories of different frames.
  arXiv  Detail & Related papers  (2026-06-04T15:49:37Z)
• SRENet: Spectral Re-Entry Network for Point Cloud Action Recognition [49.508770157706486]
  We propose SRENet to learn both global context and fine-grained temporal dynamics of motion from a frequency perspective for action recognition.<n>SRENet achieves state-of-the-art performance, validating the effectiveness of frequency modeling in point cloud-based action understanding.
  arXiv  Detail & Related papers  (2026-06-02T05:11:47Z)
• FIS-DiT: Breaking the Few-Step Video Inference Barrier via Training-Free Frame Interleaved Sparsity [23.184639887235218]
  Video Diffusion Transformers (DiTs) can be substantially reduced through model, but per-step inference latency remains a critical bottleneck.<n>We propose Frame Inter Sparsity DiT (FIS-DiT), a training-free and operator-agnostic framework that shifts the optimization focus from the temporal trajectory to the latent frame position.<n>FIS-DiT consistently achieves 2.11--2.41$times$ speedup with negligible degradation across VBench-Q and CLIP metrics.
  arXiv  Detail & Related papers  (2026-05-12T09:49:36Z)
• ST-Prune: Training-Free Spatio-Temporal Token Pruning for Vision-Language Models in Autonomous Driving [31.688411695647357]
  We propose ST-Prune, a training-free, plug-and-play framework comprising two complementary modules: Motionaware RSP (MTP) and Ring-view Spatial Pruning (RSPRSP)<n>These two modules constitute a complete-temporal pruning process, preserving key geometry-the-art for training-free token pruning.<n> ST-Prune achieves near-lossless performance with certain metrics surpassing the full-model baseline, while maintaining speeds comparable to existing pruning approaches.
  arXiv  Detail & Related papers  (2026-04-21T06:51:08Z)
• Inference-Time Temporal Probability Smoothing for Stable Video Segmentation with SAM2 under Weak Prompts [0.0]
  The proposed framework is lightweight, model-agnostic, and well-suited for real-time, interactive video segmentation.<n>We evaluate the proposed method on four diverse video sequences using a comprehensive set of frame-wise and temporal stability metrics.
  arXiv  Detail & Related papers  (2026-04-18T19:16:22Z)
• Enhancing Low-Cost Video Editing with Lightweight Adaptors and Temporal-Aware Inversion [28.38307253613529]
  We propose a framework that integrates temporal-spatial and semantic consistency with Baliteral DDIM inversion.<n>Our method significantly improves perceptual quality, text-image alignment, and temporal coherence, as demonstrated on the MSR-VTT dataset.
  arXiv  Detail & Related papers  (2025-01-08T16:41:31Z)
• Low-Light Video Enhancement via Spatial-Temporal Consistent Decomposition [52.89441679581216]
  Low-Light Video Enhancement (LLVE) seeks to restore dynamic or static scenes plagued by severe invisibility and noise.<n>We present an innovative video decomposition strategy that incorporates view-independent and view-dependent components.<n>Our framework consistently outperforms existing methods, establishing a new SOTA performance.
  arXiv  Detail & Related papers  (2024-05-24T15:56:40Z)
• Video Dynamics Prior: An Internal Learning Approach for Robust Video Enhancements [83.5820690348833]
  We present a framework for low-level vision tasks that does not require any external training data corpus. Our approach learns neural modules by optimizing over a corrupted sequence, leveraging the weights of the coherence-temporal test and statistics internal statistics.
  arXiv  Detail & Related papers  (2023-12-13T01:57:11Z)
• RIGID: Recurrent GAN Inversion and Editing of Real Face Videos [73.97520691413006]
  GAN inversion is indispensable for applying the powerful editability of GAN to real images. Existing methods invert video frames individually often leading to undesired inconsistent results over time. We propose a unified recurrent framework, named textbfRecurrent vtextbfIdeo textbfGAN textbfInversion and etextbfDiting (RIGID) Our framework learns the inherent coherence between input frames in an end-to-end manner.
  arXiv  Detail & Related papers  (2023-08-11T12:17:24Z)
• Recurrent Self-Supervised Video Denoising with Denser Receptive Field [33.3711070590966]
  Self-supervised video denoising has seen decent progress through the use of blind spot networks. Previous self-supervised video denoising methods suffer from significant information loss and texture destruction in either the whole reference frame or neighbor frames. We propose RDRF for self-supervised video denoising, which fully exploits both the reference and neighbor frames with a denser receptive field.
  arXiv  Detail & Related papers  (2023-08-07T14:09:08Z)
• TimeLens: Event-based Video Frame Interpolation [54.28139783383213]
  We introduce Time Lens, a novel indicates equal contribution method that leverages the advantages of both synthesis-based and flow-based approaches. We show an up to 5.21 dB improvement in terms of PSNR over state-of-the-art frame-based and event-based methods.
  arXiv  Detail & Related papers  (2021-06-14T10:33:47Z)
• Zooming SlowMo: An Efficient One-Stage Framework for Space-Time Video Super-Resolution [100.11355888909102]
  Space-time video super-resolution aims at generating a high-resolution (HR) slow-motion video from a low-resolution (LR) and low frame rate (LFR) video sequence. We present a one-stage space-time video super-resolution framework, which can directly reconstruct an HR slow-motion video sequence from an input LR and LFR video.
  arXiv  Detail & Related papers  (2021-04-15T17:59:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.