Strivec: Sparse Tri-Vector Radiance Fields

• URL: http://arxiv.org/abs/2307.13226v2
• Date: Thu, 24 Aug 2023 06:29:35 GMT
• Title: Strivec: Sparse Tri-Vector Radiance Fields
• Authors: Quankai Gao, Qiangeng Xu, Hao Su, Ulrich Neumann, Zexiang Xu
• Abstract summary: Strivec is a novel representation that models a 3D scene as a radiance field with sparsely distributed and compactly factorized local tensor feature grids. We demonstrate that our model can achieve better rendering quality while using significantly fewer parameters than previous methods.
• Score: 40.66438698104296
• License: http://creativecommons.org/publicdomain/zero/1.0/
• Abstract: We propose Strivec, a novel neural representation that models a 3D scene as a radiance field with sparsely distributed and compactly factorized local tensor feature grids. Our approach leverages tensor decomposition, following the recent work TensoRF, to model the tensor grids. In contrast to TensoRF which uses a global tensor and focuses on their vector-matrix decomposition, we propose to utilize a cloud of local tensors and apply the classic CANDECOMP/PARAFAC (CP) decomposition to factorize each tensor into triple vectors that express local feature distributions along spatial axes and compactly encode a local neural field. We also apply multi-scale tensor grids to discover the geometry and appearance commonalities and exploit spatial coherence with the tri-vector factorization at multiple local scales. The final radiance field properties are regressed by aggregating neural features from multiple local tensors across all scales. Our tri-vector tensors are sparsely distributed around the actual scene surface, discovered by a fast coarse reconstruction, leveraging the sparsity of a 3D scene. We demonstrate that our model can achieve better rendering quality while using significantly fewer parameters than previous methods, including TensoRF and Instant-NGP.

Related papers

• Neural Signed Distance Function Inference through Splatting 3D Gaussians Pulled on Zero-Level Set [49.780302894956776]
  It is vital to infer a signed distance function (SDF) in multi-view based surface reconstruction. We propose a method that seamlessly merge 3DGS with the learning of neural SDFs. Our numerical and visual comparisons show our superiority over the state-of-the-art results on the widely used benchmarks.
  arXiv  Detail & Related papers  (2024-10-18T05:48:06Z)
• N-BVH: Neural ray queries with bounding volume hierarchies [51.430495562430565]
  In 3D computer graphics, the bulk of a scene's memory usage is due to polygons and textures. We devise N-BVH, a neural compression architecture designed to answer arbitrary ray queries in 3D. Our method provides faithful approximations of visibility, depth, and appearance attributes.
  arXiv  Detail & Related papers  (2024-05-25T13:54:34Z)
• Gaussian Opacity Fields: Efficient Adaptive Surface Reconstruction in Unbounded Scenes [50.92217884840301]
  Gaussian Opacity Fields (GOF) is a novel approach for efficient, high-quality, and adaptive surface reconstruction in scenes. GOF is derived from ray-tracing-based volume rendering of 3D Gaussians. GOF surpasses existing 3DGS-based methods in surface reconstruction and novel view synthesis.
  arXiv  Detail & Related papers  (2024-04-16T17:57:19Z)
• pixelSplat: 3D Gaussian Splats from Image Pairs for Scalable Generalizable 3D Reconstruction [26.72289913260324]
  pixelSplat is a feed-forward model that learns to reconstruct 3D radiance fields parameterized by 3D Gaussian primitives from pairs of images. Our model features real-time and memory-efficient rendering for scalable training as well as fast 3D reconstruction at inference time.
  arXiv  Detail & Related papers  (2023-12-19T17:03:50Z)
• Tensor Decomposition Based Attention Module for Spiking Neural Networks [18.924242014716647]
  We design the textitprojected full attention (PFA) module, which demonstrates excellent results with linearly growing parameters. Our method achieves state-of-the-art performance on both static and dynamic benchmark datasets.
  arXiv  Detail & Related papers  (2023-10-23T05:25:49Z)
• Low-Rank Tensor Function Representation for Multi-Dimensional Data Recovery [52.21846313876592]
  Low-rank tensor function representation (LRTFR) can continuously represent data beyond meshgrid with infinite resolution. We develop two fundamental concepts for tensor functions, i.e., the tensor function rank and low-rank tensor function factorization. Our method substantiates the superiority and versatility of our method as compared with state-of-the-art methods.
  arXiv  Detail & Related papers  (2022-12-01T04:00:38Z)
• TensoRF: Tensorial Radiance Fields [74.16791688888081]
  We present TensoRF, a novel approach to model and reconstruct radiance fields. We model the radiance field of a scene as a 4D tensor, which represents a 3D voxel grid with per-voxel multi-channel features. We show that TensoRF with CP decomposition achieves fast reconstruction (30 min) with better rendering quality and even a smaller model size (4 MB) compared to NeRF.
  arXiv  Detail & Related papers  (2022-03-17T17:59:59Z)
• 2D+3D facial expression recognition via embedded tensor manifold regularization [16.98176664818354]
  A novel approach via embedded tensor manifold regularization for 2D+3D facial expression recognition (FERETMR) is proposed. We establish the first-order optimality condition in terms of stationary points, and then design a block coordinate descent (BCD) algorithm with convergence analysis. Numerical results on BU-3DFE database and Bosphorus databases demonstrate the effectiveness of our proposed approach.
  arXiv  Detail & Related papers  (2022-01-29T06:11:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.