Neural Fourier Filter Bank

• URL: http://arxiv.org/abs/2212.01735v4
• Date: Thu, 24 Aug 2023 04:39:38 GMT
• Title: Neural Fourier Filter Bank
• Authors: Zhijie Wu and Yuhe Jin and Kwang Moo Yi
• Abstract summary: We present a novel method to provide efficient and highly detailed reconstructions. Inspired by wavelets, we learn a neural field that decompose the signal both spatially and frequency-wise.
• Score: 18.52741992605852
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present a novel method to provide efficient and highly detailed reconstructions. Inspired by wavelets, we learn a neural field that decompose the signal both spatially and frequency-wise. We follow the recent grid-based paradigm for spatial decomposition, but unlike existing work, encourage specific frequencies to be stored in each grid via Fourier features encodings. We then apply a multi-layer perceptron with sine activations, taking these Fourier encoded features in at appropriate layers so that higher-frequency components are accumulated on top of lower-frequency components sequentially, which we sum up to form the final output. We demonstrate that our method outperforms the state of the art regarding model compactness and convergence speed on multiple tasks: 2D image fitting, 3D shape reconstruction, and neural radiance fields. Our code is available at https://github.com/ubc-vision/NFFB.

Related papers

• MDNF: Multi-Diffusion-Nets for Neural Fields on Meshes [5.284425534494986]
  We propose a novel framework for representing neural fields on triangle meshes that is multi-resolution across both spatial and frequency domains. Inspired by the Neural Fourier Filter Bank (NFFB), our architecture decomposes the frequencies and frequency domains by associating finer resolution levels with higher frequency bands. We demonstrate the effectiveness of our approach through its application to diverse neural fields, such as synthetic RGB functions, UV texture coordinates, and normals.
  arXiv  Detail & Related papers  (2024-09-04T19:08:13Z)
• FourierMamba: Fourier Learning Integration with State Space Models for Image Deraining [71.46369218331215]
  Image deraining aims to remove rain streaks from rainy images and restore clear backgrounds. We propose a new framework termed FourierMamba, which performs image deraining with Mamba in the Fourier space.
  arXiv  Detail & Related papers  (2024-05-29T18:58:59Z)
• FFPN: Fourier Feature Pyramid Network for Ultrasound Image Segmentation [15.011573950064424]
  Ultrasound (US) image segmentation is an active research area that requires real-time and highly accurate analysis in many scenarios. Existing approaches may suffer from inadequate contour encoding or fail to effectively leverage the encoded results. In this paper, we introduce a novel Fourier-anchor-based DTS framework called Fourier Feature Pyramid Network (FFPN) to address the aforementioned issues.
  arXiv  Detail & Related papers  (2023-08-26T07:28:09Z)
• Fourier-Net+: Leveraging Band-Limited Representation for Efficient 3D Medical Image Registration [62.53130123397081]
  U-Net style networks are commonly utilized in unsupervised image registration to predict dense displacement fields. We first propose Fourier-Net, which replaces the costly U-Net style expansive path with a parameter-free model-driven decoder. We then introduce Fourier-Net+, which additionally takes the band-limited spatial representation of the images as input and further reduces the number of convolutional layers in the U-Net style network's contracting path.
  arXiv  Detail & Related papers  (2023-07-06T13:57:12Z)
• Harnessing Low-Frequency Neural Fields for Few-Shot View Synthesis [82.31272171857623]
  We harness low-frequency neural fields to regularize high-frequency neural fields from overfitting. We propose a simple-yet-effective strategy for tuning the frequency to avoid overfitting few-shot inputs.
  arXiv  Detail & Related papers  (2023-03-15T05:15:21Z)
• QFF: Quantized Fourier Features for Neural Field Representations [28.82293263445964]
  We show that using Quantized Fourier Features (QFF) can result in smaller model size, faster training, and better quality outputs for several applications. QFF are easy to code, fast to compute, and serve as a simple drop-in addition to many neural field representations.
  arXiv  Detail & Related papers  (2022-12-02T00:11:22Z)
• NAF: Neural Attenuation Fields for Sparse-View CBCT Reconstruction [79.13750275141139]
  This paper proposes a novel and fast self-supervised solution for sparse-view CBCT reconstruction. The desired attenuation coefficients are represented as a continuous function of 3D spatial coordinates, parameterized by a fully-connected deep neural network. A learning-based encoder entailing hash coding is adopted to help the network capture high-frequency details.
  arXiv  Detail & Related papers  (2022-09-29T04:06:00Z)
• PREF: Phasorial Embedding Fields for Compact Neural Representations [54.44527545923917]
  We present a phasorial embedding field emphPREF as a compact representation to facilitate neural signal modeling and reconstruction tasks. Our experiments show PREF-based neural signal processing technique is on par with the state-of-the-art in 2D image completion, 3D SDF surface regression, and 5D radiance field reconstruction.
  arXiv  Detail & Related papers  (2022-05-26T17:43:03Z)
• Fourier Disentangled Space-Time Attention for Aerial Video Recognition [54.80846279175762]
  We present an algorithm, Fourier Activity Recognition (FAR), for UAV video activity recognition. Our formulation uses a novel Fourier object disentanglement method to innately separate out the human agent from the background. We have evaluated our approach on multiple UAV datasets including UAV Human RGB, UAV Human Night, Drone Action, and NEC Drone.
  arXiv  Detail & Related papers  (2022-03-21T01:24:53Z)
• Fourier PlenOctrees for Dynamic Radiance Field Rendering in Real-time [43.0484840009621]
  Implicit neural representations such as Neural Radiance Field (NeRF) have focused mainly on modeling static objects captured under multi-view settings. We present a novel Fourier PlenOctree (FPO) technique to tackle efficient neural modeling and real-time rendering of dynamic scenes captured under the free-view video (FVV) setting. We show that the proposed method is 3000 times faster than the original NeRF and over an order of magnitude acceleration over SOTA.
  arXiv  Detail & Related papers  (2022-02-17T11:57:01Z)
• Seeing Implicit Neural Representations as Fourier Series [13.216389226310987]
  Implicit Neural Representations (INR) use multilayer perceptrons to represent high-frequency functions in low-dimensional problem domains. These representations achieved state-of-the-art results on tasks related to complex 3D objects and scenes. This work analyzes the connection between the two methods and shows that a Fourier mapped perceptron is structurally like one hidden layer SIREN.
  arXiv  Detail & Related papers  (2021-09-01T08:40:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.