K-Planes: Explicit Radiance Fields in Space, Time, and Appearance

• URL: http://arxiv.org/abs/2301.10241v2
• Date: Fri, 24 Mar 2023 21:32:50 GMT
• Title: K-Planes: Explicit Radiance Fields in Space, Time, and Appearance
• Authors: Sara Fridovich-Keil, Giacomo Meanti, Frederik Warburg, Benjamin Recht, Angjoo Kanazawa
• Abstract summary: We introduce k-planes, a white-box model for radiance fields in arbitrary dimensions. Our model uses d choose 2 planes to represent a d-dimensional scene, providing a seamless way to go from static to dynamic scenes. Across a range of synthetic and real, static and dynamic, fixed and varying appearance scenes, k-planes yields competitive and often state-of-the-art reconstruction fidelity.
• Score: 32.78595254330191
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We introduce k-planes, a white-box model for radiance fields in arbitrary dimensions. Our model uses d choose 2 planes to represent a d-dimensional scene, providing a seamless way to go from static (d=3) to dynamic (d=4) scenes. This planar factorization makes adding dimension-specific priors easy, e.g. temporal smoothness and multi-resolution spatial structure, and induces a natural decomposition of static and dynamic components of a scene. We use a linear feature decoder with a learned color basis that yields similar performance as a nonlinear black-box MLP decoder. Across a range of synthetic and real, static and dynamic, fixed and varying appearance scenes, k-planes yields competitive and often state-of-the-art reconstruction fidelity with low memory usage, achieving 1000x compression over a full 4D grid, and fast optimization with a pure PyTorch implementation. For video results and code, please see https://sarafridov.github.io/K-Planes.

Related papers

• Grid4D: 4D Decomposed Hash Encoding for High-fidelity Dynamic Gaussian Splatting [21.47981274362659]
  We propose Grid4D, a dynamic scene rendering model based on Gaussian splatting. We decompose the 4D encoding into one spatial and three temporal 3D hash encodings without the low-rank assumption. Our experiments demonstrate that Grid4D significantly outperforms the state-of-the-art models in visual quality and rendering speed.
  arXiv  Detail & Related papers  (2024-10-28T08:02:34Z)
• DaRePlane: Direction-aware Representations for Dynamic Scene Reconstruction [26.39519157164198]
  We present DaRePlane, a novel representation approach that captures dynamics from six different directions. DaRePlane yields state-of-the-art performance in novel view synthesis for various complex dynamic scenes.
  arXiv  Detail & Related papers  (2024-10-18T04:19:10Z)
• TK-Planes: Tiered K-Planes with High Dimensional Feature Vectors for Dynamic UAV-based Scenes [58.180556221044235]
  We present a new approach to bridge the domain gap between synthetic and real-world data for unmanned aerial vehicle (UAV)-based perception. Our formulation is designed for dynamic scenes, consisting of small moving objects or human actions. We evaluate its performance on challenging datasets, including Okutama Action and UG2.
  arXiv  Detail & Related papers  (2024-05-04T21:55:33Z)
• Compact 3D Scene Representation via Self-Organizing Gaussian Grids [10.816451552362823]
  3D Gaussian Splatting has recently emerged as a highly promising technique for modeling of static 3D scenes. We introduce a compact scene representation organizing the parameters of 3DGS into a 2D grid with local homogeneity. Our method achieves a reduction factor of 17x to 42x in size for complex scenes with no increase in training time.
  arXiv  Detail & Related papers  (2023-12-19T20:18:29Z)
• WavePlanes: A compact Wavelet representation for Dynamic Neural Radiance Fields [9.158626732325915]
  This paper presents WavePlanes, a fast and more compact explicit model. We propose a multi-scale space and space-time feature plane representation using N-level 2-D wavelet coefficients. Exploiting the sparsity of wavelet coefficients, we compress the model using a Hash Map.
  arXiv  Detail & Related papers  (2023-12-03T15:19:08Z)
• Dynamic 3D Gaussians: Tracking by Persistent Dynamic View Synthesis [58.5779956899918]
  We present a method that simultaneously addresses the tasks of dynamic scene novel-view synthesis and six degree-of-freedom (6-DOF) tracking of all dense scene elements. We follow an analysis-by-synthesis framework, inspired by recent work that models scenes as a collection of 3D Gaussians. We demonstrate a large number of downstream applications enabled by our representation, including first-person view synthesis, dynamic compositional scene synthesis, and 4D video editing.
  arXiv  Detail & Related papers  (2023-08-18T17:59:21Z)
• MoDA: Modeling Deformable 3D Objects from Casual Videos [84.29654142118018]
  We propose neural dual quaternion blend skinning (NeuDBS) to achieve 3D point deformation without skin-collapsing artifacts. In the endeavor to register 2D pixels across different frames, we establish a correspondence between canonical feature embeddings that encodes 3D points within the canonical space. Our approach can reconstruct 3D models for humans and animals with better qualitative and quantitative performance than state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-04-17T13:49:04Z)
• HexPlane: A Fast Representation for Dynamic Scenes [18.276921637560445]
  We show that dynamic 3D scenes can be explicitly represented by six planes of learned features, leading to an elegant solution we call HexPlane. A HexPlane computes features for points in spacetime by fusing vectors extracted from each plane, which is highly efficient.
  arXiv  Detail & Related papers  (2023-01-23T18:59:25Z)
• NeRFPlayer: A Streamable Dynamic Scene Representation with Decomposed Neural Radiance Fields [99.57774680640581]
  We present an efficient framework capable of fast reconstruction, compact modeling, and streamable rendering. We propose to decompose the 4D space according to temporal characteristics. Points in the 4D space are associated with probabilities belonging to three categories: static, deforming, and new areas.
  arXiv  Detail & Related papers  (2022-10-28T07:11:05Z)
• Neural Deformable Voxel Grid for Fast Optimization of Dynamic View Synthesis [63.25919018001152]
  We propose a fast deformable radiance field method to handle dynamic scenes. Our method achieves comparable performance to D-NeRF using only 20 minutes for training.
  arXiv  Detail & Related papers  (2022-06-15T17:49:08Z)
• Learning Dynamic View Synthesis With Few RGBD Cameras [60.36357774688289]
  We propose to utilize RGBD cameras to synthesize free-viewpoint videos of dynamic indoor scenes. We generate point clouds from RGBD frames and then render them into free-viewpoint videos via a neural feature. We introduce a simple Regional Depth-Inpainting module that adaptively inpaints missing depth values to render complete novel views.
  arXiv  Detail & Related papers  (2022-04-22T03:17:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.