Modulated Periodic Activations for Generalizable Local Functional Representations

• URL: http://arxiv.org/abs/2104.03960v1
• Date: Thu, 8 Apr 2021 17:59:04 GMT
• Title: Modulated Periodic Activations for Generalizable Local Functional Representations
• Authors: Ishit Mehta, Micha\"el Gharbi, Connelly Barnes, Eli Shechtman, Ravi Ramamoorthi, Manmohan Chandraker
• Abstract summary: We present a new representation that generalizes to multiple instances and achieves state-of-the-art fidelity. Our approach produces general functional representations of images, videos and shapes, and achieves higher reconstruction quality than prior works that are optimized for a single signal.
• Score: 113.64179351957888
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Multi-Layer Perceptrons (MLPs) make powerful functional representations for sampling and reconstruction problems involving low-dimensional signals like images,shapes and light fields. Recent works have significantly improved their ability to represent high-frequency content by using periodic activations or positional encodings. This often came at the expense of generalization: modern methods are typically optimized for a single signal. We present a new representation that generalizes to multiple instances and achieves state-of-the-art fidelity. We use a dual-MLP architecture to encode the signals. A synthesis network creates a functional mapping from a low-dimensional input (e.g. pixel-position) to the output domain (e.g. RGB color). A modulation network maps a latent code corresponding to the target signal to parameters that modulate the periodic activations of the synthesis network. We also propose a local-functional representation which enables generalization. The signal's domain is partitioned into a regular grid,with each tile represented by a latent code. At test time, the signal is encoded with high-fidelity by inferring (or directly optimizing) the latent code-book. Our approach produces generalizable functional representations of images, videos and shapes, and achieves higher reconstruction quality than prior works that are optimized for a single signal.

Related papers

• FINER++: Building a Family of Variable-periodic Functions for Activating Implicit Neural Representation [39.116375158815515]
  Implicit Neural Representation (INR) is causing a revolution in the field of signal processing. INR techniques suffer from the "frequency"-specified spectral bias and capacity-convergence gap. We propose the FINER++ framework by extending existing periodic/non-periodic activation functions to variable-periodic ones.
  arXiv  Detail & Related papers  (2024-07-28T09:24:57Z)
• Towards Signal Processing In Large Language Models [46.76681147411957]
  This paper introduces the idea of applying signal processing inside a Large Language Model (LLM) We draw parallels between classical Fourier-Transforms and Fourier Transform-like learnable time-frequency representations. We show that for GPT-like architectures, our work achieves faster convergence and significantly increases performance.
  arXiv  Detail & Related papers  (2024-06-10T13:51:52Z)
• In-Domain GAN Inversion for Faithful Reconstruction and Editability [132.68255553099834]
  We propose in-domain GAN inversion, which consists of a domain-guided domain-regularized and a encoder to regularize the inverted code in the native latent space of the pre-trained GAN model. We make comprehensive analyses on the effects of the encoder structure, the starting inversion point, as well as the inversion parameter space, and observe the trade-off between the reconstruction quality and the editing property.
  arXiv  Detail & Related papers  (2023-09-25T08:42:06Z)
• Neural Fourier Filter Bank [18.52741992605852]
  We present a novel method to provide efficient and highly detailed reconstructions. Inspired by wavelets, we learn a neural field that decompose the signal both spatially and frequency-wise.
  arXiv  Detail & Related papers  (2022-12-04T03:45:08Z)
• PINs: Progressive Implicit Networks for Multi-Scale Neural Representations [68.73195473089324]
  We propose a progressive positional encoding, exposing a hierarchical structure to incremental sets of frequency encodings. Our model accurately reconstructs scenes with wide frequency bands and learns a scene representation at progressive level of detail. Experiments on several 2D and 3D datasets show improvements in reconstruction accuracy, representational capacity and training speed compared to baselines.
  arXiv  Detail & Related papers  (2022-02-09T20:33:37Z)
• Meta-Learning Sparse Implicit Neural Representations [69.15490627853629]
  Implicit neural representations are a promising new avenue of representing general signals. Current approach is difficult to scale for a large number of signals or a data set. We show that meta-learned sparse neural representations achieve a much smaller loss than dense meta-learned models.
  arXiv  Detail & Related papers  (2021-10-27T18:02:53Z)
• Seeing Implicit Neural Representations as Fourier Series [13.216389226310987]
  Implicit Neural Representations (INR) use multilayer perceptrons to represent high-frequency functions in low-dimensional problem domains. These representations achieved state-of-the-art results on tasks related to complex 3D objects and scenes. This work analyzes the connection between the two methods and shows that a Fourier mapped perceptron is structurally like one hidden layer SIREN.
  arXiv  Detail & Related papers  (2021-09-01T08:40:20Z)
• Learnable Fourier Features for Multi-DimensionalSpatial Positional Encoding [96.9752763607738]
  We propose a novel positional encoding method based on learnable Fourier features. Our experiments show that our learnable feature representation for multi-dimensional positional encoding outperforms existing methods.
  arXiv  Detail & Related papers  (2021-06-05T04:40:18Z)
• Learned Initializations for Optimizing Coordinate-Based Neural Representations [47.408295381897815]
  Coordinate-based neural representations have shown significant promise as an alternative to discrete, array-based representations. We propose applying standard meta-learning algorithms to learn the initial weight parameters for these fully-connected networks. We explore these benefits across a variety of tasks, including representing 2D images, reconstructing CT scans, and recovering 3D shapes and scenes from 2D image observations.
  arXiv  Detail & Related papers  (2020-12-03T18:59:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.