Toward Semantic Communication for Real-time Mobile 3D Reconstruction

• URL: http://arxiv.org/abs/2607.16128v1
• Date: Fri, 17 Jul 2026 17:07:40 GMT
• Title: Toward Semantic Communication for Real-time Mobile 3D Reconstruction
• Abstract summary: We propose a SemCom framework for real-time mobile 3D reconstruction.<n>The framework includes a semantic transceiver that outputs a reconstructed image alongside a pixel-wise confidence map.<n>Our framework significantly improves pose estimation accuracy and 3D structural consistency.
• Score: 6.297166067940033
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Real-time mobile 3D reconstruction is fundamental to many emerging applications such as autonomous navigation and digital twin construction, where a moving platform continuously captures an image stream and transmit to a computing server for scene understanding. Unlike offline reconstruction, camera poses and scene geometry are estimated on-the-fly during acquisition, making multi-view consistency a real-time requirement and rendering geometric estimation highly sensitive to communication-induced distortions. Semantic communication (SemCom) transmits compact semantic information, offering a promising way to preserve task-critical data over unreliable links. However, existing designs are optimized at the image or single-view level and without providing explicit reliability information for geometric estimation, limiting their applicability to real-time mobile 3D reconstruction. In this context, we propose a SemCom framework for real-time mobile 3D reconstruction. The framework includes a semantic transceiver that outputs a reconstructed image alongside a pixel-wise confidence map, quantifying the reliability of each region. We further introduce a confidence-guided geometric estimation method, incorporating confidence into RANSAC-based pose initialization and bundle adjustment to reduce the influence of unreliable regions and enhance robustness under noisy channels. Simulations show that, compared to existing SemCom and traditional seperate source and channel coding, our framework maintains high image quality while significantly improving pose estimation accuracy and 3D structural consistency.

Related papers

• GAN-Based Semantic Communication for Image Transmission in IoV [6.095348773114754]
  This paper proposes a generative adversarial network-based semantic communication framework to address the efficiency and fidelity bottlenecks of traditional communication systems.<n>At the transmitter, the framework adopts a pyramid attention network to extract semantic label maps and introduces a semantic priority preservation mechanism.<n>At the receiver, an image reconstruction module integrates a coarse to-fine multi-resolution generator and multi-scale discriminator.<n>The proposed method outperforms existing counterparts in both semantic segmentation accuracy and reconstructed image quality.
  arXiv  Detail & Related papers  (2026-08-28T06:58:19Z)
• Pano3D: Unified 3D Reconstruction and Panoptic Segmentation [68.18993232814061]
  We introduce an approach that performs 3D reconstruction and 3D panoptic segmentation in a unified framework.<n>We build on existing 3D reconstruction models and augment them with a set-based mask decoder.<n>Our method achieves state-of-the-art performance in 3D panoptic segmentation across ScanNet, ScanNet200, and ScanNet++ datasets.
  arXiv  Detail & Related papers  (2026-06-12T09:43:13Z)
• Geometric Context Transformer for Streaming 3D Reconstruction [51.19524805829903]
  LingBot-Map is a feed-forward 3D foundation model for reconstructing scenes from streaming data.<n>A defining aspect of LingBot-Map lies in its carefully designed attention mechanism, which integrates an anchor context, a pose-reference window, and a trajectory memory.<n>This design keeps the streaming state compact while retaining rich geometric context, enabling stable efficient inference at around 20 FPS on 518 x 378 resolution inputs.
  arXiv  Detail & Related papers  (2026-04-15T17:58:13Z)
• Reliev3R: Relieving Feed-forward Reconstruction from Multi-View Geometric Annotations [98.66466590444553]
  We propose Reliev3R, a weakly-supervised paradigm for training FFRMs from scratch without cost-prohibitive multi-view geometric annotations.<n>At the core of Reliev3R, we design an ambiguity-aware relative depth loss and a trigonometry-based reprojection loss to facilitate supervision for multi-view geometric consistency.
  arXiv  Detail & Related papers  (2026-04-01T06:46:54Z)
• Follow My Hold: Hand-Object Interaction Reconstruction through Geometric Guidance [61.41904916189093]
  We propose a novel diffusion-based framework for reconstructing 3D geometry of hand-held objects from monocular RGB images.<n>We use hand-object interaction as geometric guidance to ensure plausible hand-object interactions.
  arXiv  Detail & Related papers  (2025-08-25T17:11:53Z)
• 3D Reconstruction via Incremental Structure From Motion [1.4999444543328293]
  We present a detailed implementation of the incremental SfM pipeline, focusing on the consistency of geometric estimation and the effect of iterative refinement through bundle adjustment.<n>Results support the practical utility of incremental SfM as a reliable method for sparse 3D reconstruction in visually structured environments.
  arXiv  Detail & Related papers  (2025-08-01T18:45:05Z)
• MoNetV2: Enhanced Motion Network for Freehand 3D Ultrasound Reconstruction [11.531888235029445]
  We propose an enhanced motion network (MoNetV2) to enhance the accuracy and generalizability of reconstruction under diverse scanning velocities and tactics.<n>MoNetV2 surpasses existing methods in both reconstruction quality and generalizability performance across three large datasets.
  arXiv  Detail & Related papers  (2025-06-16T04:57:34Z)
• MUSt3R: Multi-view Network for Stereo 3D Reconstruction [11.61182864709518]
  We propose an extension of DUSt3R from pairs to multiple views, that addresses all aforementioned concerns.<n>We entail the model with a multi-layer memory mechanism which allows to reduce the computational complexity.<n>The framework is designed to perform 3D reconstruction both offline and online, and hence can be seamlessly applied to SfM and visual SLAM scenarios.
  arXiv  Detail & Related papers  (2025-03-03T15:36:07Z)
• Evolving High-Quality Rendering and Reconstruction in a Unified Framework with Contribution-Adaptive Regularization [27.509109317973817]
  3D Gaussian Splatting (3DGS) has garnered significant attention for its high-quality rendering and fast inference speed.<n>Previous methods primarily focus on geometry regularization, with common approaches including primitive-based and dual-model frameworks.<n>We propose CarGS, a unified model leveraging-adaptive regularization to achieve simultaneous, high-quality surface reconstruction.
  arXiv  Detail & Related papers  (2025-03-02T12:51:38Z)
• MaRINeR: Enhancing Novel Views by Matching Rendered Images with Nearby References [49.71130133080821]
  MaRINeR is a refinement method that leverages information of a nearby mapping image to improve the rendering of a target viewpoint. We show improved renderings in quantitative metrics and qualitative examples from both explicit and implicit scene representations.
  arXiv  Detail & Related papers  (2024-07-18T17:50:03Z)
• Deep Learning-based Cross-modal Reconstruction of Vehicle Target from Sparse 3D SAR Image [6.499547636078961]
  We introduce cross-modal learning and propose a Cross-Modal 3D-SAR Reconstruction Network (CMAR-Net) for enhancing sparse 3D SAR images of vehicle targets by fusing optical information.<n>CMAR-Net achieves efficient training and reconstructs sparse 3D SAR images, which are derived from highly sparse-aspect observations, into visually structured 3D vehicle images.
  arXiv  Detail & Related papers  (2024-06-06T15:18:59Z)
• SCFusion: Real-time Incremental Scene Reconstruction with Semantic Completion [86.77318031029404]
  We propose a framework that performs scene reconstruction and semantic scene completion jointly in an incremental and real-time manner. Our framework relies on a novel neural architecture designed to process occupancy maps and leverages voxel states to accurately and efficiently fuse semantic completion with the 3D global model.
  arXiv  Detail & Related papers  (2020-10-26T15:31:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.