PNeRV: Enhancing Spatial Consistency via Pyramidal Neural Representation for Videos

• URL: http://arxiv.org/abs/2404.08921v1
• Date: Sat, 13 Apr 2024 07:50:17 GMT
• Title: PNeRV: Enhancing Spatial Consistency via Pyramidal Neural Representation for Videos
• Abstract summary: We introduce the Pyramidal Neural Representation for Videos (PNeRV) It is built on a multi-scale information connection and comprises a lightweight rescaling operator, Kronecker Fully-connected layer (KFc), and a Selective Memory (BSM) mechanism. PNeRV achieves a +4.49 dB gain in PSNR and a 231% increase in FVD on UVG, along with a +3.28 dB PSNR and 634% FVD increase on DAVIS.
• Score: 40.94458611212317
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The primary focus of Neural Representation for Videos (NeRV) is to effectively model its spatiotemporal consistency. However, current NeRV systems often face a significant issue of spatial inconsistency, leading to decreased perceptual quality. To address this issue, we introduce the Pyramidal Neural Representation for Videos (PNeRV), which is built on a multi-scale information connection and comprises a lightweight rescaling operator, Kronecker Fully-connected layer (KFc), and a Benign Selective Memory (BSM) mechanism. The KFc, inspired by the tensor decomposition of the vanilla Fully-connected layer, facilitates low-cost rescaling and global correlation modeling. BSM merges high-level features with granular ones adaptively. Furthermore, we provide an analysis based on the Universal Approximation Theory of the NeRV system and validate the effectiveness of the proposed PNeRV.We conducted comprehensive experiments to demonstrate that PNeRV surpasses the performance of contemporary NeRV models, achieving the best results in video regression on UVG and DAVIS under various metrics (PSNR, SSIM, LPIPS, and FVD). Compared to vanilla NeRV, PNeRV achieves a +4.49 dB gain in PSNR and a 231% increase in FVD on UVG, along with a +3.28 dB PSNR and 634% FVD increase on DAVIS.

Related papers

• tcnerv:dual-domain temporal context modeling for implicit neural video compression [6.025390826523129]
  TCNeRV exploits reconstructed context in both feature and embedding domains.<n>It achieves an average PSNR of 36.08 dB on the UVG dataset, outperforming HNeRV-Boost by 2.20 dB.
  arXiv  Detail & Related papers  (2026-09-15T09:00:18Z)
• TinyNeRV: Compact Neural Video Representations via Capacity Scaling, Distillation, and Low-Precision Inference [2.294014185517203]
  Implicit neural video representations encode entire video sequences within the parameters of a neural network.<n>Recent work on Neural Representations for Videos (NeRV) has demonstrated competitive reconstruction performance.<n>This paper presents a systematic study of tiny NeRV architectures designed for efficient deployment.
  arXiv  Detail & Related papers  (2026-04-10T11:26:09Z)
• LRConv-NeRV: Low Rank Convolution for Efficient Neural Video Compression [2.7920304852537527]
  This paper proposes LRConv-NeRV, an efficient NeRV variant that replaces selected dense 3x3 convolutional layers with structured low-rank separable convolutions.<n>Experiments demonstrate that applying LRConv only to the final decoder stage reduces decoder complexity by 68%.<n>Results establish LRConv-NeRV as a potential architectural alternative for efficient neural video decoding under low-precision and resource-constrained settings.
  arXiv  Detail & Related papers  (2026-03-18T20:31:59Z)
• Rethinking Diffusion Model-Based Video Super-Resolution: Leveraging Dense Guidance from Aligned Features [51.5076190312734]
  Video Super-Resolution approaches suffer from error accumulation, spatial artifacts, and a trade-off between perceptual quality and fidelity.<n>We propose a novelly Guided diffusion model with Aligned Features for Video Super-Resolution (DGAF-VSR)<n>Experiments on synthetic and real-world datasets demonstrate that DGAF-VSR surpasses state-of-the-art methods in key aspects of VSR.
  arXiv  Detail & Related papers  (2025-11-21T03:40:45Z)
• NIRVANA: Structured pruning reimagined for large language models compression [50.651730342011014]
  We introduce NIRVANA, a novel pruning method designed to balance immediate zero-shot preservation accuracy with robust fine-tuning.<n>To further address the unique challenges posed by structured pruning, NIRVANA incorporates an adaptive sparsity allocation mechanism across layers and modules.<n>Experiments conducted on Llama3, Qwen, T5 models demonstrate that NIRVANA outperforms existing structured pruning methods under equivalent sparsity constraints.
  arXiv  Detail & Related papers  (2025-09-17T17:59:00Z)
• FANeRV: Frequency Separation and Augmentation based Neural Representation for Video [32.38933743785333]
  We present a Frequency Separation and Augmentation based Neural Representation for video (FANeRV) FANeRV explicitly separates input frames into high and low-frequency components using discrete wavelet transform. A specially designed gated network effectively fuses these frequency components for optimal reconstruction.
  arXiv  Detail & Related papers  (2025-04-09T10:19:35Z)
• VRVVC: Variable-Rate NeRF-Based Volumetric Video Compression [59.14355576912495]
  NeRF-based video has revolutionized visual media by delivering photorealistic Free-Viewpoint Video (FVV) experiences. The substantial data volumes pose significant challenges for storage and transmission. We propose VRVVC, a novel end-to-end joint variable-rate framework for video compression.
  arXiv  Detail & Related papers  (2024-12-16T01:28:04Z)
• INN-PAR: Invertible Neural Network for PPG to ABP Reconstruction [9.127220498800645]
  We introduce an invertible neural network for PPG to ABP reconstruction (INN-PAR) INN-PAR efficiently captures both forward and inverse mappings simultaneously, thereby preventing information loss. We propose a multi-scale convolution module (MSCM) within the invertible block, enabling the model to learn features across multiple scales effectively.
  arXiv  Detail & Related papers  (2024-09-13T17:48:48Z)
• PNeRV: A Polynomial Neural Representation for Videos [28.302862266270093]
  Extracting Implicit Neural Representations on video poses unique challenges due to the additional temporal dimension. We introduce Polynomial Neural Representation for Videos (PNeRV) PNeRV mitigates challenges posed by video data in the realm of INRs but opens new avenues for advanced video processing and analysis.
  arXiv  Detail & Related papers  (2024-06-27T16:15:22Z)
• VQ-NeRV: A Vector Quantized Neural Representation for Videos [3.6662666629446043]
  Implicit neural representations (INR) excel in encoding videos within neural networks, showcasing promise in computer vision tasks like video compression and denoising. We introduce an advanced U-shaped architecture, Vector Quantized-NeRV (VQ-NeRV), which integrates a novel component--the VQ-NeRV Block. This block incorporates a codebook mechanism to discretize the network's shallow residual features and inter-frame residual information effectively.
  arXiv  Detail & Related papers  (2024-03-19T03:19:07Z)
• NERV++: An Enhanced Implicit Neural Video Representation [11.25130799452367]
  We introduce neural representations for videos NeRV++, an enhanced implicit neural video representation. NeRV++ is more straightforward yet effective enhancement over the original NeRV decoder architecture. We evaluate our method on UVG, MCL JVC, and Bunny datasets, achieving competitive results for video compression with INRs.
  arXiv  Detail & Related papers  (2024-02-28T13:00:32Z)
• Using BOLD-fMRI to Compute the Respiration Volume per Time (RTV) and Respiration Variation (RV) with Convolutional Neural Networks (CNN) in the Human Connectome Development Cohort [55.41644538483948]
  This study proposes a one-dimensional CNN model for reconstruction of two respiratory measures, RV and RVT. Results show that a CNN can capture informative features from resting BOLD signals and reconstruct realistic RV and RVT timeseries.
  arXiv  Detail & Related papers  (2023-07-03T18:06:36Z)
• Camera-Based HRV Prediction for Remote Learning Environments [4.074837550066978]
  Restoring blood volume pulse signals from facial videos is a challenging task that involves a series of preprocessing, image algorithms, and postprocessing to restore waveforms. The challenge in obtaining HRV indices through r is the necessity for algorithms to precisely predict the BVP peak positions. In this paper, we collected the Remote Learning Affect and Physiology (RLAP) dataset, which includes over 32 hours of highly synchronized video and labels from 58 subjects. Using the RLAP dataset, we trained a new model called Seq-r, it is a model based on one-dimensional convolution, and experimental results reveal
  arXiv  Detail & Related papers  (2023-05-07T02:26:00Z)
• DNeRV: Modeling Inherent Dynamics via Difference Neural Representation for Videos [53.077189668346705]
  Difference Representation for Videos (eRV) We analyze this from the perspective of limitation function fitting and the importance of frame difference. DNeRV achieves competitive results against the state-of-the-art neural compression approaches.
  arXiv  Detail & Related papers  (2023-04-13T13:53:49Z)
• Distortion-Aware Loop Filtering of Intra 360^o Video Coding with Equirectangular Projection [81.63407194858854]
  We propose a distortion-aware loop filtering model to improve the performance of intra coding for 360$o$ videos projected via equirectangular projection (ERP) format. Our proposed module analyzes content characteristics based on a coding unit (CU) partition mask and processes them through partial convolution to activate the specified area.
  arXiv  Detail & Related papers  (2022-02-20T12:00:18Z)
• RetiNerveNet: Using Recursive Deep Learning to Estimate Pointwise 24-2 Visual Field Data based on Retinal Structure [109.33721060718392]
  glaucoma is the leading cause of irreversible blindness in the world, affecting over 70 million people. Due to the Standard Automated Perimetry (SAP) test's innate difficulty and its high test-retest variability, we propose the RetiNerveNet.
  arXiv  Detail & Related papers  (2020-10-15T03:09:08Z)
• Iterative Network for Image Super-Resolution [69.07361550998318]
  Single image super-resolution (SISR) has been greatly revitalized by the recent development of convolutional neural networks (CNN) This paper provides a new insight on conventional SISR algorithm, and proposes a substantially different approach relying on the iterative optimization. A novel iterative super-resolution network (ISRN) is proposed on top of the iterative optimization.
  arXiv  Detail & Related papers  (2020-05-20T11:11:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.