Neural-MMGS: Multi-modal Neural Gaussian Splats for Large-Scale Scene Reconstruction

• URL: http://arxiv.org/abs/2509.17762v1
• Date: Mon, 22 Sep 2025 13:24:58 GMT
• Title: Neural-MMGS: Multi-modal Neural Gaussian Splats for Large-Scale Scene Reconstruction
• Abstract summary: We propose Neural-MMGS, a framework for multimodal large-scale scene reconstruction.<n>Our approach fuses all modalities -- image, LiDAR, and semantics -- into a compact, learnable embedding.<n>We evaluate this framework on the Oxford Spires and KITTI-360 datasets.
• Score: 7.923706570755559
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper proposes Neural-MMGS, a novel neural 3DGS framework for multimodal large-scale scene reconstruction that fuses multiple sensing modalities in a per-gaussian compact, learnable embedding. While recent works focusing on large-scale scene reconstruction have incorporated LiDAR data to provide more accurate geometric constraints, we argue that LiDAR's rich physical properties remain underexplored. Similarly, semantic information has been used for object retrieval, but could provide valuable high-level context for scene reconstruction. Traditional approaches append these properties to Gaussians as separate parameters, increasing memory usage and limiting information exchange across modalities. Instead, our approach fuses all modalities -- image, LiDAR, and semantics -- into a compact, learnable embedding that implicitly encodes optical, physical, and semantic features in each Gaussian. We then train lightweight neural decoders to map these embeddings to Gaussian parameters, enabling the reconstruction of each sensing modality with lower memory overhead and improved scalability. We evaluate Neural-MMGS on the Oxford Spires and KITTI-360 datasets. On Oxford Spires, we achieve higher-quality reconstructions, while on KITTI-360, our method reaches competitive results with less storage consumption compared with current approaches in LiDAR-based novel-view synthesis.

Related papers

• Is Gaussian Splatting Becoming Neural Again? A Taxonomy and Controlled Study of Learned Parameterization [6.732693952041468]
  Three-dimensional Gaussian Splatting (3DGS) combines explicit primitives with efficientization.<n>Recent systems increasingly use neural networks to generate or share Gaussian parameters.<n>We characterize this trend along five axes: attribute decoding, spatial sharing, view-conditioned decoding, topology generation, and amortized inference.
  arXiv  Detail & Related papers  (2026-09-11T03:37:31Z)
• Neural Texture Splatting: Expressive 3D Gaussian Splatting for View Synthesis, Geometry, and Dynamic Reconstruction [20.80508604651125]
  3D Gaussian Splatting (3DGS) has emerged as a leading approach for high-quality novel view synthesis.<n>Recent works have proposed to augment 3DGS with additional per-primitive capacity, such as per-splat textures.<n>We introduce Neural Texture Splatting (NTS) to improve state-of-the-art 3DGS variants across a wide range of reconstruction tasks.
  arXiv  Detail & Related papers  (2025-11-24T08:26:32Z)
• Detail Across Scales: Multi-Scale Enhancement for Full Spectrum Neural Representations [4.899720537787801]
  Implicit neural representations (INRs) have emerged as a compact and parametric alternative to discrete array-based data representations.<n>We propose WIEN-INR, a wavelet-informed implicit neural representation that distributes modeling across different resolution scales.<n>We show that WIEN-INR achieves superior reconstruction fidelity while maintaining a compact model size.
  arXiv  Detail & Related papers  (2025-09-19T00:15:39Z)
• MuGS: Multi-Baseline Generalizable Gaussian Splatting Reconstruction [13.941042770932794]
  We present Multi-Baseline Gaussian Splatting (MuRF), a feed-forward approach for novel view synthesis.<n>MuRF achieves state-of-the-art performance across multiple baseline settings and diverse scenarios.
  arXiv  Detail & Related papers  (2025-08-06T10:34:24Z)
• HoliGS: Holistic Gaussian Splatting for Embodied View Synthesis [59.25751939710903]
  We propose a novel deformable Gaussian splatting framework that addresses embodied view synthesis from long monocular RGB videos.<n>Our method leverages invertible Gaussian Splatting deformation networks to reconstruct large-scale, dynamic environments accurately.<n>Results highlight a practical and scalable solution for EVS in real-world scenarios.
  arXiv  Detail & Related papers  (2025-06-24T03:54:40Z)
• Discretized Gaussian Representation for Tomographic Reconstruction [36.64964395405005]
  Discretized Gaussian Representation (DGR) is a novel framework that reconstructs the 3D volume directly using a set of discretized Gaussian functions.<n>Fast Volume Reconstruction is a highly parallelized technique that aggregates Gaussian contributions into the voxel grid with minimal overhead.
  arXiv  Detail & Related papers  (2024-11-07T16:32:29Z)
• SA-GS: Semantic-Aware Gaussian Splatting for Large Scene Reconstruction with Geometry Constrain [43.80789481557894]
  We propose a novel method, named SA-GS, for fine-grained 3D geometry reconstruction using semantic-aware 3D Gaussian Splats. We leverage prior information stored in large vision models such as SAM and DINO to generate semantic masks. We extract the point cloud using a novel probability density-based extraction method, transforming Gaussian Splats into a point cloud crucial for downstream tasks.
  arXiv  Detail & Related papers  (2024-05-27T08:15:10Z)
• N-BVH: Neural ray queries with bounding volume hierarchies [51.430495562430565]
  In 3D computer graphics, the bulk of a scene's memory usage is due to polygons and textures. We devise N-BVH, a neural compression architecture designed to answer arbitrary ray queries in 3D. Our method provides faithful approximations of visibility, depth, and appearance attributes.
  arXiv  Detail & Related papers  (2024-05-25T13:54:34Z)
• HAC: Hash-grid Assisted Context for 3D Gaussian Splatting Compression [55.6351304553003]
  3D Gaussian Splatting (3DGS) has emerged as a promising framework for novel view synthesis. We propose a Hash-grid Assisted Context (HAC) framework for highly compact 3DGS representation. Our work is the pioneer to explore context-based compression for 3DGS representation, resulting in a remarkable size reduction of over $75times$ compared to vanilla 3DGS.
  arXiv  Detail & Related papers  (2024-03-21T16:28:58Z)
• Neural Kernel Surface Reconstruction [80.51581494300423]
  We present a novel method for reconstructing a 3D implicit surface from a large-scale, sparse, and noisy point cloud. Our approach builds upon the recently introduced Neural Kernel Fields representation.
  arXiv  Detail & Related papers  (2023-05-31T06:25:18Z)
• NeRF-LOAM: Neural Implicit Representation for Large-Scale Incremental LiDAR Odometry and Mapping [14.433784957457632]
  We propose a novel NeRF-based LiDAR odometry and mapping approach, NeRF-LOAM, consisting of three modules neural odometry, neural mapping, and mesh reconstruction. Our approach achieves state-of-the-art odometry and mapping performance, as well as a strong generalization in large-scale environments utilizing LiDAR data.
  arXiv  Detail & Related papers  (2023-03-19T16:40:36Z)
• VolRecon: Volume Rendering of Signed Ray Distance Functions for Generalizable Multi-View Reconstruction [64.09702079593372]
  VolRecon is a novel generalizable implicit reconstruction method with Signed Ray Distance Function (SRDF) On DTU dataset, VolRecon outperforms SparseNeuS by about 30% in sparse view reconstruction and achieves comparable accuracy as MVSNet in full view reconstruction.
  arXiv  Detail & Related papers  (2022-12-15T18:59:54Z)
• BNV-Fusion: Dense 3D Reconstruction using Bi-level Neural Volume Fusion [85.24673400250671]
  We present Bi-level Neural Volume Fusion (BNV-Fusion), which leverages recent advances in neural implicit representations and neural rendering for dense 3D reconstruction. In order to incrementally integrate new depth maps into a global neural implicit representation, we propose a novel bi-level fusion strategy. We evaluate the proposed method on multiple datasets quantitatively and qualitatively, demonstrating a significant improvement over existing methods.
  arXiv  Detail & Related papers  (2022-04-03T19:33:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.