Neural Field Tokenizations with Hierarchy and Spatial Locality Priors

• URL: http://arxiv.org/abs/2606.08204v1
• Date: Sat, 06 Jun 2026 14:43:30 GMT
• Title: Neural Field Tokenizations with Hierarchy and Spatial Locality Priors
• Abstract summary: We propose LH-NeF, a framework to learn general-purpose tokenized representations of continuous signals.<n>By replacing meta-learning's inner loop with a single forward pass, LH-NeF uses 42$times$ less memory.<n>Across images, 3D shapes, and climate fields, our learned representations match or exceed performance of modality-agnostic, modality-specific, and specialized generative neural field baselines.
• Score: 26.386588932734345
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Neural fields parameterize data as functions from coordinates to values, providing a unified framework for representation learning across modalities. Existing approaches are dominated by per-sample meta-learning, which scales poorly due to memory-intensive inner-loop optimization. The natural alternative -- feed-forward encoding -- typically introduces modality-specific assumptions, sacrificing the generality that makes learning with neural fields attractive. We argue that locality and hierarchy are useful priors for learning field representations that can be injected without compromising modality-agnosticism. We propose LH-NeF, a framework to learn general-purpose tokenized representations of continuous signals. A locality-preserving hierarchical encoder maps raw coordinate-value field observations to structured tokens, from which the field is reconstructed during training. By replacing meta-learning's inner loop with a single forward pass, LH-NeF uses 42$\times$ less memory and supports 133$\times$ larger batches than the strongest modality-agnostic baseline. Across images, 3D shapes, and climate fields, our learned representations match or exceed performance of modality-agnostic, modality-specific, and specialized generative neural field baselines on both reconstruction and downstream tasks.

Related papers

• Stochastic Preconditioning for Neural Field Optimization [32.78919701189794]
  This work observes that fitting neural fields is greatly improved by incorporating spatiality during training.<n>A simple technique can replace or even outperform custom-designed hierarchies and frequency space constructions.
  arXiv  Detail & Related papers  (2025-05-26T19:13:41Z)
• Clustering Inductive Biases with Unrolled Networks [4.47196217712431]
  We propose an autoencoder architecture (WLSC) whose latent representations are implicitly, locally organized for spectral clustering through a Laplacian quadratic form of a bipartite graph. We show that our regularization can be interpreted as early-stage specialization of receptive fields to certain classes of stimuli.
  arXiv  Detail & Related papers  (2023-11-30T02:02:30Z)
• Neural Processing of Tri-Plane Hybrid Neural Fields [20.78031512517053]
  We show that the tri-plane discrete data structure encodes rich information, which can be effectively processed by standard deep-learning machinery. While processing a field with the same reconstruction quality, we achieve task performance far superior to frameworks that process larges representations.
  arXiv  Detail & Related papers  (2023-10-02T12:27:22Z)
• Generalizable Neural Fields as Partially Observed Neural Processes [16.202109517569145]
  We propose a new paradigm that views the large-scale training of neural representations as a part of a partially-observed neural process framework. We demonstrate that this approach outperforms both state-of-the-art gradient-based meta-learning approaches and hypernetwork approaches.
  arXiv  Detail & Related papers  (2023-09-13T01:22:16Z)
• SimNP: Learning Self-Similarity Priors Between Neural Points [52.4201466988562]
  SimNP is a method to learn category-level self-similarities. We show that SimNP is able to outperform previous methods in reconstructing symmetric unseen object regions.
  arXiv  Detail & Related papers  (2023-09-07T16:02:40Z)
• General Neural Gauge Fields [100.35916421218101]
  We develop a learning framework to jointly optimize gauge transformations and neural fields. We derive an information-invariant gauge transformation which allows to preserve scene information inherently and yield superior performance.
  arXiv  Detail & Related papers  (2023-05-05T12:08:57Z)
• Sparse, Geometric Autoencoder Models of V1 [2.491226380993217]
  We propose an autoencoder architecture whose latent representations are implicitly, locally organized for spectral clustering. We show that the autoencoder objective function maintains core ideas of the sparse coding framework, yet also offers a promising path to describe the differentiation of receptive fields.
  arXiv  Detail & Related papers  (2023-02-22T06:07:20Z)
• Learning Large-scale Neural Fields via Context Pruned Meta-Learning [60.93679437452872]
  We introduce an efficient optimization-based meta-learning technique for large-scale neural field training. We show how gradient re-scaling at meta-test time allows the learning of extremely high-quality neural fields. Our framework is model-agnostic, intuitive, straightforward to implement, and shows significant reconstruction improvements for a wide range of signals.
  arXiv  Detail & Related papers  (2023-02-01T17:32:16Z)
• Meta-Learning Sparse Implicit Neural Representations [69.15490627853629]
  Implicit neural representations are a promising new avenue of representing general signals. Current approach is difficult to scale for a large number of signals or a data set. We show that meta-learned sparse neural representations achieve a much smaller loss than dense meta-learned models.
  arXiv  Detail & Related papers  (2021-10-27T18:02:53Z)
• PredRNN: A Recurrent Neural Network for Spatiotemporal Predictive Learning [109.84770951839289]
  We present PredRNN, a new recurrent network for learning visual dynamics from historical context. We show that our approach obtains highly competitive results on three standard datasets.
  arXiv  Detail & Related papers  (2021-03-17T08:28:30Z)
• MetaSDF: Meta-learning Signed Distance Functions [85.81290552559817]
  Generalizing across shapes with neural implicit representations amounts to learning priors over the respective function space. We formalize learning of a shape space as a meta-learning problem and leverage gradient-based meta-learning algorithms to solve this task.
  arXiv  Detail & Related papers  (2020-06-17T05:14:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.