FANeRV: Frequency Separation and Augmentation based Neural Representation for Video

• URL: http://arxiv.org/abs/2504.06755v4
• Date: Tue, 13 May 2025 09:42:01 GMT
• Title: FANeRV: Frequency Separation and Augmentation based Neural Representation for Video
• Abstract summary: We present a Frequency Separation and Augmentation based Neural Representation for video (FANeRV)<n>FANeRV explicitly separates input frames into high and low-frequency components using discrete wavelet transform.<n>A specially designed gated network effectively fuses these frequency components for optimal reconstruction.
• Score: 32.35716293561769
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Neural representations for video (NeRV) have gained considerable attention for their strong performance across various video tasks. However, existing NeRV methods often struggle to capture fine spatial details, resulting in vague reconstructions. In this paper, we present a Frequency Separation and Augmentation based Neural Representation for video (FANeRV), which addresses these limitations with its core Wavelet Frequency Upgrade Block. This block explicitly separates input frames into high and low-frequency components using discrete wavelet transform, followed by targeted enhancement using specialized modules. Finally, a specially designed gated network effectively fuses these frequency components for optimal reconstruction. Additionally, convolutional residual enhancement blocks are integrated into the later stages of the network to balance parameter distribution and improve the restoration of high-frequency details. Experimental results demonstrate that FANeRV significantly improves reconstruction performance and excels in multiple tasks, including video compression, inpainting, and interpolation, outperforming existing NeRV methods.

Related papers

• SoLAR: Error-Resilient Streamable Long-Horizon Free-Viewpoint Video Reconstruction with Anchor Activation and Latent Recalibration [57.159190580279585]
  Motivated by bit allocation theory, we analyze dynamic-anchor-based volumetric video representation within a rate-distortion optimization framework.<n>We propose textbfSoLAR, which is the first error-resilient streamable FVV framework that maintains stable reconstruction quality on long sequences.
  arXiv  Detail & Related papers  (2026-05-08T06:48:59Z)
• Frequency-aware Neural Representation for Videos [31.60687845071296]
  We propose FaNeRV, a Frequency-aware Neural Representation for videos.<n>FaNeRV explicitly decouples low- and high-frequency components to enable efficient and faithful video reconstruction.<n>Experiments on standard benchmarks demonstrate that FaNeRV significantly outperforms state-of-the-art INR methods.
  arXiv  Detail & Related papers  (2026-01-25T08:19:13Z)
• Rethinking Diffusion Model-Based Video Super-Resolution: Leveraging Dense Guidance from Aligned Features [51.5076190312734]
  Video Super-Resolution approaches suffer from error accumulation, spatial artifacts, and a trade-off between perceptual quality and fidelity.<n>We propose a novelly Guided diffusion model with Aligned Features for Video Super-Resolution (DGAF-VSR)<n>Experiments on synthetic and real-world datasets demonstrate that DGAF-VSR surpasses state-of-the-art methods in key aspects of VSR.
  arXiv  Detail & Related papers  (2025-11-21T03:40:45Z)
• DiTVR: Zero-Shot Diffusion Transformer for Video Restoration [48.97196894658511]
  DiTVR is a zero shot video restoration framework that couples a diffusion transformer with trajectory aware attention and a flow consistent sampler.<n>Our attention mechanism aligns tokens along optical flow trajectories, with particular emphasis on vital layers that exhibit the highest sensitivity to temporal dynamics.<n>The flow guided sampler injects data consistency only into low-frequency bands, preserving high frequency priors while accelerating cache.
  arXiv  Detail & Related papers  (2025-08-11T09:54:45Z)
• FADPNet: Frequency-Aware Dual-Path Network for Face Super-Resolution [70.61549422952193]
  Face super-resolution (FSR) under limited computational costs remains an open problem.<n>Existing approaches typically treat all facial pixels equally, resulting in suboptimal allocation of computational resources.<n>We propose FADPNet, a Frequency-Aware Dual-Path Network that decomposes facial features into low- and high-frequency components.
  arXiv  Detail & Related papers  (2025-06-17T02:33:42Z)
• SR-NeRV: Improving Embedding Efficiency of Neural Video Representation via Super-Resolution [0.0]
  Implicit Neural Representations (INRs) have garnered significant attention for their ability to model complex signals in various domains.<n>We propose an INR-based video representation framework that integrates a general-purpose super-resolution (SR) network.<n>By offloading the reconstruction of fine details to a dedicated SR network pre-trained on natural images, the proposed method improves visual fidelity.
  arXiv  Detail & Related papers  (2025-04-30T03:31:40Z)
• Rethinking Video Tokenization: A Conditioned Diffusion-based Approach [58.164354605550194]
  New tokenizer, Diffusion Conditioned-based Gene Tokenizer, replaces GAN-based decoder with conditional diffusion model. We trained using only a basic MSE diffusion loss for reconstruction, along with KL term and LPIPS perceptual loss from scratch. Even a scaled-down version of CDT (3$times inference speedup) still performs comparably with top baselines.
  arXiv  Detail & Related papers  (2025-03-05T17:59:19Z)
• SNeRV: Spectra-preserving Neural Representation for Video [8.978061470104532]
  We propose spectra-preserving NeRV (SNeRV) as a novel approach to enhance implicit video representations.<n>In this paper, we use 2D discrete wavelet transform (DWT) to decompose video into low-frequency (LF) and high-frequency (HF) features.<n>We demonstrate that SNeRV outperforms existing NeRV models in capturing fine details and achieves enhanced reconstruction.
  arXiv  Detail & Related papers  (2025-01-03T07:57:38Z)
• VRVVC: Variable-Rate NeRF-Based Volumetric Video Compression [59.14355576912495]
  NeRF-based video has revolutionized visual media by delivering photorealistic Free-Viewpoint Video (FVV) experiences.<n>The substantial data volumes pose significant challenges for storage and transmission.<n>We propose VRVVC, a novel end-to-end joint variable-rate framework for video compression.
  arXiv  Detail & Related papers  (2024-12-16T01:28:04Z)
• High-Frequency Enhanced Hybrid Neural Representation for Video Compression [32.38933743785333]
  This paper introduces a High-Frequency Enhanced Hybrid Neural Representation Network. Our method focuses on leveraging high-frequency information to improve the synthesis of fine details by the network. Experiments on the Bunny and UVG datasets demonstrate that our method outperforms other methods.
  arXiv  Detail & Related papers  (2024-11-11T03:04:46Z)
• Few-shot NeRF by Adaptive Rendering Loss Regularization [78.50710219013301]
  Novel view synthesis with sparse inputs poses great challenges to Neural Radiance Field (NeRF) Recent works demonstrate that the frequency regularization of Positional rendering can achieve promising results for few-shot NeRF. We propose Adaptive Rendering loss regularization for few-shot NeRF, dubbed AR-NeRF.
  arXiv  Detail & Related papers  (2024-10-23T13:05:26Z)
• Neural Video Representation for Redundancy Reduction and Consistency Preservation [0.0]
  Implicit neural representation (INR) embed various signals into neural networks. We propose a video representation method that generates both the high-frequency and low-frequency components of the frame. Experimental results demonstrate that our method outperforms the existing HNeRV method, achieving superior results in 96 percent of the videos.
  arXiv  Detail & Related papers  (2024-09-27T07:30:12Z)
• VQ-NeRV: A Vector Quantized Neural Representation for Videos [3.6662666629446043]
  Implicit neural representations (INR) excel in encoding videos within neural networks, showcasing promise in computer vision tasks like video compression and denoising. We introduce an advanced U-shaped architecture, Vector Quantized-NeRV (VQ-NeRV), which integrates a novel component--the VQ-NeRV Block. This block incorporates a codebook mechanism to discretize the network's shallow residual features and inter-frame residual information effectively.
  arXiv  Detail & Related papers  (2024-03-19T03:19:07Z)
• Boosting Neural Representations for Videos with a Conditional Decoder [28.073607937396552]
  Implicit neural representations (INRs) have emerged as a promising approach for video storage and processing. This paper introduces a universal boosting framework for current implicit video representation approaches.
  arXiv  Detail & Related papers  (2024-02-28T08:32:19Z)
• Neural Residual Radiance Fields for Streamably Free-Viewpoint Videos [69.22032459870242]
  We present a novel technique, Residual Radiance Field or ReRF, as a highly compact neural representation to achieve real-time free-view rendering on long-duration dynamic scenes. We show such a strategy can handle large motions without sacrificing quality. Based on ReRF, we design a special FVV that achieves three orders of magnitudes compression rate and provides a companion ReRF player to support online streaming of long-duration FVVs of dynamic scenes.
  arXiv  Detail & Related papers  (2023-04-10T08:36:00Z)
• Structured Sparsity Learning for Efficient Video Super-Resolution [99.1632164448236]
  We develop a structured pruning scheme called Structured Sparsity Learning (SSL) according to the properties of video super-resolution (VSR) models. In SSL, we design pruning schemes for several key components in VSR models, including residual blocks, recurrent networks, and upsampling networks.
  arXiv  Detail & Related papers  (2022-06-15T17:36:04Z)
• Multi-level Wavelet-based Generative Adversarial Network for Perceptual Quality Enhancement of Compressed Video [51.631731922593225]
  Existing methods mainly focus on enhancing the objective quality of compressed video while ignoring its perceptual quality. We propose a novel generative adversarial network (GAN) based on multi-level wavelet packet transform (WPT) to enhance the perceptual quality of compressed video.
  arXiv  Detail & Related papers  (2020-08-02T15:01:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.