Adaptive Time-step Training for Enhancing Spike-Based Neural Radiance Fields

• URL: http://arxiv.org/abs/2507.23033v1
• Date: Wed, 30 Jul 2025 18:56:24 GMT
• Title: Adaptive Time-step Training for Enhancing Spike-Based Neural Radiance Fields
• Authors: Ranxi Lin, Canming Yao, Jiayi Li, Weihang Liu, Xin Lou, Pingqiang Zhou,
• Abstract summary: We propose a spike-based NeRF framework with a dynamic time step training strategy, termed Pretrain-Adaptive Time-step Adjustment (PATA)<n>We show that PATA can preserve rendering fidelity while reducing inference time steps by 64% and running power by 61.55%.
• Score: 6.66530903309279
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Neural Radiance Fields (NeRF)-based models have achieved remarkable success in 3D reconstruction and rendering tasks. However, during both training and inference, these models rely heavily on dense point sampling along rays from multiple viewpoints, resulting in a surge in floating-point operations and severely limiting their use in resource-constrained scenarios like edge computing. Spiking Neural Networks (SNNs), which communicate via binary spikes over discrete time steps, offer a promising alternative due to their energy-efficient nature. Given the inherent variability in scene scale and texture complexity in neural rendering and the prevailing practice of training separate models per scene, we propose a spike-based NeRF framework with a dynamic time step training strategy, termed Pretrain-Adaptive Time-step Adjustment (PATA). This approach automatically explores the trade-off between rendering quality and time step length during training. Consequently, it enables scene-adaptive inference with variable time steps and reduces the additional consumption of computational resources in the inference process. Anchoring to the established Instant-NGP architecture, we evaluate our method across diverse datasets. The experimental results show that PATA can preserve rendering fidelity while reducing inference time steps by 64\% and running power by 61.55\%.

Related papers

• A Stable Whitening Optimizer for Efficient Neural Network Training [101.89246340672246]
  Building on the Shampoo family of algorithms, we identify and alleviate three key issues, resulting in the proposed SPlus method.<n>First, we find that naive Shampoo is prone to divergence when matrix-inverses are cached for long periods.<n>Second, we adapt a shape-aware scaling to enable learning rate transfer across network width.<n>Third, we find that high learning rates result in large parameter noise, and propose a simple iterate-averaging scheme which unblocks faster learning.
  arXiv  Detail & Related papers  (2025-06-08T18:43:31Z)
• STLight: a Fully Convolutional Approach for Efficient Predictive Learning by Spatio-Temporal joint Processing [6.872340834265972]
  We propose STLight, a novel method for S-temporal learning that relies solely on channel-wise and depth-wise convolutions as learnable layers. STLight overcomes the limitations of traditional convolutional approaches by rearranging spatial and temporal dimensions together. Our architecture achieves state-of-the-art performance on STL benchmarks across datasets and settings, while significantly improving computational efficiency in terms of parameters and computational FLOPs.
  arXiv  Detail & Related papers  (2024-11-15T13:53:19Z)
• Trajectory Flow Matching with Applications to Clinical Time Series Modeling [77.58277281319253]
  Trajectory Flow Matching (TFM) trains a Neural SDE in a simulation-free manner, bypassing backpropagation through the dynamics.<n>We demonstrate improved performance on three clinical time series datasets in terms of absolute performance and uncertainty prediction.
  arXiv  Detail & Related papers  (2024-10-28T15:54:50Z)
• Efficient NeRF Optimization -- Not All Samples Remain Equally Hard [9.404889815088161]
  We propose an application of online hard sample mining for efficient training of Neural Radiance Fields (NeRF) NeRF models produce state-of-the-art quality for many 3D reconstruction and rendering tasks but require substantial computational resources.
  arXiv  Detail & Related papers  (2024-08-06T13:49:01Z)
• Estimating Post-Synaptic Effects for Online Training of Feed-Forward SNNs [0.27016900604393124]
  Facilitating online learning in spiking neural networks (SNNs) is a key step in developing event-based models. We propose Online Training with Postsynaptic Estimates (OTPE) for training feed-forward SNNs. We show improved scaling for multi-layer networks using a novel approximation of temporal effects on the subsequent layer's activity.
  arXiv  Detail & Related papers  (2023-11-07T16:53:39Z)
• SpikingNeRF: Making Bio-inspired Neural Networks See through the Real World [19.696976370895907]
  We propose SpikingNeRF, which aligns the temporal dimension of spiking neural networks (SNNs) with the radiance rays. The computation turns into a spike-based, multiplication-free manner, reducing energy consumption and making high-quality 3D rendering accessible to neuromorphic hardware.
  arXiv  Detail & Related papers  (2023-09-20T01:04:57Z)
• Towards Memory- and Time-Efficient Backpropagation for Training Spiking Neural Networks [70.75043144299168]
  Spiking Neural Networks (SNNs) are promising energy-efficient models for neuromorphic computing. We propose the Spatial Learning Through Time (SLTT) method that can achieve high performance while greatly improving training efficiency. Our method achieves state-of-the-art accuracy on ImageNet, while the memory cost and training time are reduced by more than 70% and 50%, respectively, compared with BPTT.
  arXiv  Detail & Related papers  (2023-02-28T05:01:01Z)
• Gait Recognition in the Wild with Multi-hop Temporal Switch [81.35245014397759]
  gait recognition in the wild is a more practical problem that has attracted the attention of the community of multimedia and computer vision. This paper presents a novel multi-hop temporal switch method to achieve effective temporal modeling of gait patterns in real-world scenes.
  arXiv  Detail & Related papers  (2022-09-01T10:46:09Z)
• An optimised deep spiking neural network architecture without gradients [7.183775638408429]
  We present an end-to-end trainable modular event-driven neural architecture that uses local synaptic and threshold adaptation rules. The architecture represents a highly abstracted model of existing Spiking Neural Network (SNN) architectures.
  arXiv  Detail & Related papers  (2021-09-27T05:59:12Z)
• Scene Synthesis via Uncertainty-Driven Attribute Synchronization [52.31834816911887]
  This paper introduces a novel neural scene synthesis approach that can capture diverse feature patterns of 3D scenes. Our method combines the strength of both neural network-based and conventional scene synthesis approaches.
  arXiv  Detail & Related papers  (2021-08-30T19:45:07Z)
• Predicting Training Time Without Training [120.92623395389255]
  We tackle the problem of predicting the number of optimization steps that a pre-trained deep network needs to converge to a given value of the loss function. We leverage the fact that the training dynamics of a deep network during fine-tuning are well approximated by those of a linearized model. We are able to predict the time it takes to fine-tune a model to a given loss without having to perform any training.
  arXiv  Detail & Related papers  (2020-08-28T04:29:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.