Frequency-Modulated Point Cloud Rendering with Easy Editing

• URL: http://arxiv.org/abs/2303.07596v2
• Date: Sat, 18 Mar 2023 08:37:47 GMT
• Title: Frequency-Modulated Point Cloud Rendering with Easy Editing
• Authors: Yi Zhang, Xiaoyang Huang, Bingbing Ni, Teng Li, Wenjun Zhang
• Abstract summary: We develop an effective point cloud rendering pipeline for novel view synthesis. Our pipeline supports real-time rendering and user-friendly editing. In contrast to implicit rendering, our pipeline supports high-fidelity interactive editing based on point cloud manipulation.
• Score: 46.299091125921656
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We develop an effective point cloud rendering pipeline for novel view synthesis, which enables high fidelity local detail reconstruction, real-time rendering and user-friendly editing. In the heart of our pipeline is an adaptive frequency modulation module called Adaptive Frequency Net (AFNet), which utilizes a hypernetwork to learn the local texture frequency encoding that is consecutively injected into adaptive frequency activation layers to modulate the implicit radiance signal. This mechanism improves the frequency expressive ability of the network with richer frequency basis support, only at a small computational budget. To further boost performance, a preprocessing module is also proposed for point cloud geometry optimization via point opacity estimation. In contrast to implicit rendering, our pipeline supports high-fidelity interactive editing based on point cloud manipulation. Extensive experimental results on NeRF-Synthetic, ScanNet, DTU and Tanks and Temples datasets demonstrate the superior performances achieved by our method in terms of PSNR, SSIM and LPIPS, in comparison to the state-of-the-art.

Related papers

• Frequency-aware Neural Representation for Videos [31.60687845071296]
  We propose FaNeRV, a Frequency-aware Neural Representation for videos.<n>FaNeRV explicitly decouples low- and high-frequency components to enable efficient and faithful video reconstruction.<n>Experiments on standard benchmarks demonstrate that FaNeRV significantly outperforms state-of-the-art INR methods.
  arXiv  Detail & Related papers  (2026-01-25T08:19:13Z)
• FRBNet: Revisiting Low-Light Vision through Frequency-Domain Radial Basis Network [7.386546521017689]
  We revisit low-light image formation and extend the classical Lambertian model to better characterize low-light conditions.<n>We propose a novel and end-to-end trainable module named textbfFrequency-domain textbfRadial textbfBasis textbfNetwork.<n>As a plug-and-play module, FRBNet can be integrated into existing networks for low-light downstream tasks without modifying loss functions.
  arXiv  Detail & Related papers  (2025-10-27T15:46:07Z)
• Spatial Frequency Modulation for Semantic Segmentation [32.67776930209082]
  High-frequency components are vulnerable to aliasing or distortion when propagating through downsampling layers such as strided-convolution.<n>Here, we propose a novel Spatial Frequency Modulation (SFM) that modulates high-frequency features to a lower frequency before downsampling and then demodulate them back during upsampling.<n>We also propose Multi-Scale Adaptive Upsampling (MSAU) to demodulate the modulated feature and recover high-frequency information through non-uniform upsampling.
  arXiv  Detail & Related papers  (2025-07-16T04:15:53Z)
• FADPNet: Frequency-Aware Dual-Path Network for Face Super-Resolution [70.61549422952193]
  Face super-resolution (FSR) under limited computational costs remains an open problem.<n>Existing approaches typically treat all facial pixels equally, resulting in suboptimal allocation of computational resources.<n>We propose FADPNet, a Frequency-Aware Dual-Path Network that decomposes facial features into low- and high-frequency components.
  arXiv  Detail & Related papers  (2025-06-17T02:33:42Z)
• Freqformer: Image-Demoiréing Transformer via Efficient Frequency Decomposition [83.40450475728792]
  We present Freqformer, a Transformer-based framework specifically designed for image demoir'eing through targeted frequency separation.<n>Our method performs an effective frequency decomposition that explicitly splits moir'e patterns into high-frequency spatially-localized textures and low-frequency scale-robust color distortions.<n>Experiments on various demoir'eing benchmarks demonstrate that Freqformer achieves state-of-the-art performance with a compact model size.
  arXiv  Detail & Related papers  (2025-05-25T12:23:10Z)
• SNeRV: Spectra-preserving Neural Representation for Video [8.978061470104532]
  We propose spectra-preserving NeRV (SNeRV) as a novel approach to enhance implicit video representations. In this paper, we use 2D discrete wavelet transform (DWT) to decompose video into low-frequency (LF) and high-frequency (HF) features. We demonstrate that SNeRV outperforms existing NeRV models in capturing fine details and achieves enhanced reconstruction.
  arXiv  Detail & Related papers  (2025-01-03T07:57:38Z)
• GMT: Enhancing Generalizable Neural Rendering via Geometry-Driven Multi-Reference Texture Transfer [40.70828307740121]
  Novel view synthesis (NVS) aims to generate images at arbitrary viewpoints using multi-view images, and recent insights from neural radiance fields (NeRF) have contributed to remarkable improvements. G-NeRF still struggles in representing fine details for a specific scene due to the absence of per-scene optimization. We propose a Geometry-driven Multi-reference Texture transfer network (GMT) available as a plug-and-play module designed for G-NeRF.
  arXiv  Detail & Related papers  (2024-10-01T13:30:51Z)
• Spatial Annealing for Efficient Few-shot Neural Rendering [73.49548565633123]
  We introduce an accurate and efficient few-shot neural rendering method named textbfSpatial textbfAnnealing regularized textbfNeRF (textbfSANeRF) By adding merely one line of code, SANeRF delivers superior rendering quality and much faster reconstruction speed compared to current few-shot neural rendering methods.
  arXiv  Detail & Related papers  (2024-06-12T02:48:52Z)
• NeRF-VPT: Learning Novel View Representations with Neural Radiance Fields via View Prompt Tuning [63.39461847093663]
  We propose NeRF-VPT, an innovative method for novel view synthesis to address these challenges. Our proposed NeRF-VPT employs a cascading view prompt tuning paradigm, wherein RGB information gained from preceding rendering outcomes serves as instructive visual prompts for subsequent rendering stages. NeRF-VPT only requires sampling RGB data from previous stage renderings as priors at each training stage, without relying on extra guidance or complex techniques.
  arXiv  Detail & Related papers  (2024-03-02T22:08:10Z)
• PNeRFLoc: Visual Localization with Point-based Neural Radiance Fields [54.8553158441296]
  We propose a novel visual localization framework, ie, PNeRFLoc, based on a unified point-based representation. On the one hand, PNeRFLoc supports the initial pose estimation by matching 2D and 3D feature points. On the other hand, it also enables pose refinement with novel view synthesis using rendering-based optimization.
  arXiv  Detail & Related papers  (2023-12-17T08:30:00Z)
• Re-ReND: Real-time Rendering of NeRFs across Devices [56.081995086924216]
  Re-ReND is designed to achieve real-time performance by converting the NeRF into a representation that can be efficiently processed by standard graphics pipelines. We find that Re-ReND can achieve over a 2.6-fold increase in rendering speed versus the state-of-the-art without perceptible losses in quality.
  arXiv  Detail & Related papers  (2023-03-15T15:59:41Z)
• DPFNet: A Dual-branch Dilated Network with Phase-aware Fourier Convolution for Low-light Image Enhancement [1.2645663389012574]
  Low-light image enhancement is a classical computer vision problem aiming to recover normal-exposure images from low-light images. convolutional neural networks commonly used in this field are good at sampling low-frequency local structural features in the spatial domain. We propose a novel module using the Fourier coefficients, which can recover high-quality texture details under the constraint of semantics in the frequency phase.
  arXiv  Detail & Related papers  (2022-09-16T13:56:09Z)
• Fourier PlenOctrees for Dynamic Radiance Field Rendering in Real-time [43.0484840009621]
  Implicit neural representations such as Neural Radiance Field (NeRF) have focused mainly on modeling static objects captured under multi-view settings. We present a novel Fourier PlenOctree (FPO) technique to tackle efficient neural modeling and real-time rendering of dynamic scenes captured under the free-view video (FVV) setting. We show that the proposed method is 3000 times faster than the original NeRF and over an order of magnitude acceleration over SOTA.
  arXiv  Detail & Related papers  (2022-02-17T11:57:01Z)
• Functional Regularization for Reinforcement Learning via Learned Fourier Features [98.90474131452588]
  We propose a simple architecture for deep reinforcement learning by embedding inputs into a learned Fourier basis. We show that it improves the sample efficiency of both state-based and image-based RL.
  arXiv  Detail & Related papers  (2021-12-06T18:59:52Z)
• NeRF in detail: Learning to sample for view synthesis [104.75126790300735]
  Neural radiance fields (NeRF) methods have demonstrated impressive novel view synthesis. In this work we address a clear limitation of the vanilla coarse-to-fine approach -- that it is based on a performance and not trained end-to-end for the task at hand. We introduce a differentiable module that learns to propose samples and their importance for the fine network, and consider and compare multiple alternatives for its neural architecture.
  arXiv  Detail & Related papers  (2021-06-09T17:59:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.