A Semantic Communication System for Real-time 3D Reconstruction Tasks

• URL: http://arxiv.org/abs/2412.01191v1
• Date: Mon, 02 Dec 2024 06:50:05 GMT
• Title: A Semantic Communication System for Real-time 3D Reconstruction Tasks
• Abstract summary: This paper proposes a method to achieve real-time semantic mapping tasks with limited-resource mobile devices.<n> Specifically, we design an encoding-decoding semantic communication framework for real-time semantic mapping tasks.<n>In terms of simulation experiments, based on the TUM dataset, it was verified that the system has an error of less than 0.1%.
• Score: 7.082386535888168
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: 3D semantic maps have played an increasingly important role in high-precision robot localization and scene understanding. However, real-time construction of semantic maps requires mobile edge devices with extremely high computing power, which are expensive and limit the widespread application of semantic mapping. In order to address this limitation, inspired by cloud-edge collaborative computing and the high transmission efficiency of semantic communication, this paper proposes a method to achieve real-time semantic mapping tasks with limited-resource mobile devices. Specifically, we design an encoding-decoding semantic communication framework for real-time semantic mapping tasks under limited-resource situations. In addition, considering the impact of different channel conditions on communication, this paper designs a module based on the attention mechanism to achieve stable data transmission under various channel conditions. In terms of simulation experiments, based on the TUM dataset, it was verified that the system has an error of less than 0.1% compared to the groundtruth in mapping and localization accuracy and is superior to some novel semantic communication algorithms in real-time performance and channel adaptation. Besides, we implement a prototype system to verify the effectiveness of the proposed framework and designed module in real indoor scenarios. The results show that our system can complete real-time semantic mapping tasks for common indoor objects (chairs, computers, people, etc.) with a limited-resource device, and the mapping update time is less than 1 second.

Related papers

• A VAE-Driven Multi-Task Satellite-Aided Semantic Communication Framework for 6G-Enabled Connected Autonomous Vehicles [51.449502491030756]
  We present a multi-task semantic communication framework for satellite-assisted autonomous driving.<n>The framework is trained end-to-end to jointly optimize both tasks.<n>Results show that the proposed approach achieves significant bandwidth reduction of up to 87.23% to 98.17%.
  arXiv  Detail & Related papers  (2026-07-15T06:49:41Z)
• Deep Semantic Inference over the Air: An Efficient Task-Oriented Communication System [8.459189717032647]
  We explore a deep learning-based task-oriented communication framework that jointly considers classification performance, computational latency, and communication cost.<n>We adopt ResNets-based models and evaluate them on the CIFAR-10 and CIFAR-100 datasets to simulate real-world classification tasks in wireless environments.<n>With appropriate model partitioning and semantic feature compression, the system can retain over 85% of baseline accuracy while significantly reducing both computational load and communication overhead.
  arXiv  Detail & Related papers  (2025-08-18T09:18:07Z)
• Topology-Aware Modeling for Unsupervised Simulation-to-Reality Point Cloud Recognition [63.55828203989405]
  We introduce a novel Topology-Aware Modeling (TAM) framework for Sim2Real UDA on object point clouds.<n>Our approach mitigates the domain gap by leveraging global spatial topology, characterized by low-level, high-frequency 3D structures.<n>We propose an advanced self-training strategy that combines cross-domain contrastive learning with self-training.
  arXiv  Detail & Related papers  (2025-06-26T11:53:59Z)
• Task-Oriented Semantic Communication in Large Multimodal Models-based Vehicle Networks [55.32199894495722]
  We investigate an LMM-based vehicle AI assistant using a Large Language and Vision Assistant (LLaVA)<n>To reduce computational demands and shorten response time, we optimize LLaVA's image slicing to selectively focus on areas of utmost interest to users.<n>We construct a Visual Question Answering (VQA) dataset for traffic scenarios to evaluate effectiveness.
  arXiv  Detail & Related papers  (2025-05-05T07:18:47Z)
• DP-LET: An Efficient Spatio-Temporal Network Traffic Prediction Framework [13.65226228907662]
  DP-LET is an efficient feature-temporal network traffic prediction framework. DP-LET consists of a data processing module, a local feature enhancement module, and a Transformer-based prediction module. A real-world cellular traffic prediction demonstrates the practicality of DP-LET.
  arXiv  Detail & Related papers  (2025-04-04T02:52:43Z)
• Take What You Need: Flexible Multi-Task Semantic Communications with Channel Adaptation [51.53221300103261]
  This article introduces a novel channel-adaptive and multi-task-aware semantic communication framework based on a masked auto-encoder architecture. A channel-aware extractor is employed to dynamically select relevant information in response to real-time channel conditions. Experimental results demonstrate the superior performance of our framework compared to conventional methods in tasks such as image reconstruction and object detection.
  arXiv  Detail & Related papers  (2025-02-12T09:01:25Z)
• Real-Time Metric-Semantic Mapping for Autonomous Navigation in Outdoor Environments [18.7565126823704]
  We introduce an online metric-semantic mapping system that generates a global metric-semantic mesh map of large-scale outdoor environments.<n>Our mapping process achieves exceptional speed, with frame processing taking less than 7ms, regardless of scenario scale.<n>We integrate the resultant map into a real-world navigation system, enabling metric-semantic-based terrain assessment and autonomous point-to-point navigation within a campus environment.
  arXiv  Detail & Related papers  (2024-11-30T00:05:10Z)
• Multimodal generative semantic communication based on latent diffusion model [13.035207938169844]
  This paper introduces a multimodal generative semantic communication framework named mm-GESCO. The framework ingests streams of visible and infrared modal image data, generates fused semantic segmentation maps, and transmits them. At the receiving end, the framework can reconstruct the original multimodal images based on the semantic maps.
  arXiv  Detail & Related papers  (2024-08-10T06:23:41Z)
• Agent-driven Generative Semantic Communication with Cross-Modality and Prediction [57.335922373309074]
  We propose a novel agent-driven generative semantic communication framework based on reinforcement learning. In this work, we develop an agent-assisted semantic encoder with cross-modality capability, which can track the semantic changes, channel condition, to perform adaptive semantic extraction and sampling. The effectiveness of the designed models has been verified using the UA-DETRAC dataset, demonstrating the performance gains of the overall A-GSC framework.
  arXiv  Detail & Related papers  (2024-04-10T13:24:27Z)
• Reasoning with the Theory of Mind for Pragmatic Semantic Communication [62.87895431431273]
  A pragmatic semantic communication framework is proposed in this paper. It enables effective goal-oriented information sharing between two-intelligent agents. Numerical evaluations demonstrate the framework's ability to achieve efficient communication with a reduced amount of bits.
  arXiv  Detail & Related papers  (2023-11-30T03:36:19Z)
• Rate-Adaptive Coding Mechanism for Semantic Communications With Multi-Modal Data [23.597759255020296]
  We propose a distributed multi-modal semantic communication framework incorporating the conventional channel encoder/decoder. We establish a general rate-adaptive coding mechanism for various types of multi-modal semantic tasks. Numerical results show that the proposed mechanism fares better than both conventional communication and existing semantic communication systems.
  arXiv  Detail & Related papers  (2023-05-18T07:31:37Z)
• DWRSeg: Rethinking Efficient Acquisition of Multi-scale Contextual Information for Real-time Semantic Segmentation [10.379708894083217]
  We propose a highly efficient multi-scale feature extraction method, which decomposes the original single-step method into two steps, Region Residualization-Semantic Residualization. We achieve an mIoU of 72.7% on the Cityscapes test set at a speed of 319.5 FPS on one NVIDIA GeForce GTX 1080 Ti card, which exceeds the latest methods of a speed of 69.5 FPS and 0.8% mIoU.
  arXiv  Detail & Related papers  (2022-12-02T13:55:41Z)
• Semantic-Native Communication: A Simplicial Complex Perspective [50.099494681671224]
  We study semantic communication from a topological space perspective. A transmitter first maps its data into a $k$-order simplicial complex and then learns its high-order correlations. The receiver decodes the structure and infers the missing or distorted data.
  arXiv  Detail & Related papers  (2022-10-30T22:33:44Z)
• Neuro-Symbolic Artificial Intelligence (AI) for Intent based Semantic Communication [85.06664206117088]
  6G networks must consider semantics and effectiveness (at end-user) of the data transmission. NeSy AI is proposed as a pillar for learning causal structure behind the observed data. GFlowNet is leveraged for the first time in a wireless system to learn the probabilistic structure which generates the data.
  arXiv  Detail & Related papers  (2022-05-22T07:11:57Z)
• SCFusion: Real-time Incremental Scene Reconstruction with Semantic Completion [86.77318031029404]
  We propose a framework that performs scene reconstruction and semantic scene completion jointly in an incremental and real-time manner. Our framework relies on a novel neural architecture designed to process occupancy maps and leverages voxel states to accurately and efficiently fuse semantic completion with the 3D global model.
  arXiv  Detail & Related papers  (2020-10-26T15:31:52Z)
• Multi-scale Interaction for Real-time LiDAR Data Segmentation on an Embedded Platform [62.91011959772665]
  Real-time semantic segmentation of LiDAR data is crucial for autonomously driving vehicles. Current approaches that operate directly on the point cloud use complex spatial aggregation operations. We propose a projection-based method, called Multi-scale Interaction Network (MINet), which is very efficient and accurate.
  arXiv  Detail & Related papers  (2020-08-20T19:06:11Z)
• Robust and Interpretable Grounding of Spatial References with Relation Networks [40.42540299023808]
  Learning representations of spatial references in natural language is a key challenge in tasks like autonomous navigation and robotic manipulation. Recent work has investigated various neural architectures for learning multi-modal representations for spatial concepts. We develop effective models for understanding spatial references in text that are robust and interpretable.
  arXiv  Detail & Related papers  (2020-05-02T04:11:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.