Emergent communication for AR
- URL: http://arxiv.org/abs/2308.07342v1
- Date: Sat, 12 Aug 2023 16:45:39 GMT
- Title: Emergent communication for AR
- Authors: Ruxiao Chen, Shuaishuai Guo
- Abstract summary: We propose an emergent semantic communication framework to learn the communication protocols in mobile augmented reality (MAR)
Specifically, we train two agents through a modified Lewis signaling game to emerge a discrete communication protocol spontaneously.
Experiments have shown that the proposed scheme has better generalization on unseen objects than traditional object recognition used in MAR.
- Score: 11.867942569137059
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Mobile augmented reality (MAR) is widely acknowledged as one of the
ubiquitous interfaces to the digital twin and Metaverse, demanding unparalleled
levels of latency, computational power, and energy efficiency. The existing
solutions for realizing MAR combine multiple technologies like edge, cloud
computing, and fifth-generation (5G) networks. However, the inherent
communication latency of visual data imposes apparent limitations on the
quality of experience (QoE). To address the challenge, we propose an emergent
semantic communication framework to learn the communication protocols in MAR.
Specifically, we train two agents through a modified Lewis signaling game to
emerge a discrete communication protocol spontaneously. Based on this protocol,
two agents can communicate about the abstract idea of visual data through
messages with extremely small data sizes in a noisy channel, which leads to
message errors. To better simulate real-world scenarios, we incorporate channel
uncertainty into our training process. Experiments have shown that the proposed
scheme has better generalization on unseen objects than traditional object
recognition used in MAR and can effectively enhance communication efficiency
through the utilization of small-size messages.
Related papers
- Semantic Communication for Cooperative Perception using HARQ [51.148203799109304]
We leverage an importance map to distill critical semantic information, introducing a cooperative perception semantic communication framework.
To counter the challenges posed by time-varying multipath fading, our approach incorporates the use of frequency-division multiplexing (OFDM) along with channel estimation and equalization strategies.
We introduce a novel semantic error detection method that is integrated with our semantic communication framework in the spirit of hybrid automatic repeated request (HARQ)
arXiv Detail & Related papers (2024-08-29T08:53:26Z) - Context-aware Communication for Multi-agent Reinforcement Learning [6.109127175562235]
We develop a context-aware communication scheme for multi-agent reinforcement learning (MARL)
In the first stage, agents exchange coarse representations in a broadcast fashion, providing context for the second stage.
Following this, agents utilize attention mechanisms in the second stage to selectively generate messages personalized for the receivers.
To evaluate the effectiveness of CACOM, we integrate it with both actor-critic and value-based MARL algorithms.
arXiv Detail & Related papers (2023-12-25T03:33:08Z) - Multi-Agent Reinforcement Learning Based on Representational
Communication for Large-Scale Traffic Signal Control [13.844458247041711]
Traffic signal control (TSC) is a challenging problem within intelligent transportation systems.
We propose a communication-based MARL framework for large-scale TSC.
Our framework allows each agent to learn a communication policy that dictates "which" part of the message is sent "to whom"
arXiv Detail & Related papers (2023-10-03T21:06:51Z) - Generative AI-aided Joint Training-free Secure Semantic Communications
via Multi-modal Prompts [89.04751776308656]
This paper proposes a GAI-aided SemCom system with multi-model prompts for accurate content decoding.
In response to security concerns, we introduce the application of covert communications aided by a friendly jammer.
arXiv Detail & Related papers (2023-09-05T23:24:56Z) - AC2C: Adaptively Controlled Two-Hop Communication for Multi-Agent
Reinforcement Learning [4.884877440051105]
We propose a novel communication protocol called Adaptively Controlled Two-Hop Communication (AC2C)
AC2C employs an adaptive two-hop communication strategy to enable long-range information exchange among agents to boost performance.
We evaluate AC2C on three cooperative multi-agent tasks, and the experimental results show that it outperforms relevant baselines with lower communication costs.
arXiv Detail & Related papers (2023-02-24T09:00:34Z) - Semantic Communication Enabling Robust Edge Intelligence for
Time-Critical IoT Applications [87.05763097471487]
This paper aims to design robust Edge Intelligence using semantic communication for time-critical IoT applications.
We analyze the effect of image DCT coefficients on inference accuracy and propose the channel-agnostic effectiveness encoding for offloading.
arXiv Detail & Related papers (2022-11-24T20:13:17Z) - Data-Driven Blind Synchronization and Interference Rejection for Digital
Communication Signals [98.95383921866096]
We study the potential of data-driven deep learning methods for separation of two communication signals from an observation of their mixture.
We show that capturing high-resolution temporal structures (nonstationarities) leads to substantial performance gains.
We propose a domain-informed neural network (NN) design that is able to improve upon both "off-the-shelf" NNs and classical detection and interference rejection methods.
arXiv Detail & Related papers (2022-09-11T14:10:37Z) - Learning Practical Communication Strategies in Cooperative Multi-Agent
Reinforcement Learning [5.539117319607963]
Communication in realistic wireless networks can be highly unreliable due to network conditions varying with agents' mobility.
We propose a framework to learn practical communication strategies by addressing three fundamental questions.
We show significant improvements in game performance, convergence speed and communication efficiency compared with state-of-the-art.
arXiv Detail & Related papers (2022-09-02T22:18:43Z) - Multi-agent Communication with Graph Information Bottleneck under
Limited Bandwidth (a position paper) [92.11330289225981]
In many real-world scenarios, communication can be expensive and the bandwidth of the multi-agent system is subject to certain constraints.
Redundant messages who occupy the communication resources can block the transmission of informative messages and thus jeopardize the performance.
We propose a novel multi-agent communication module, CommGIB, which effectively compresses the structure information and node information in the communication graph to deal with bandwidth-constrained settings.
arXiv Detail & Related papers (2021-12-20T07:53:44Z) - Communication-Efficient and Distributed Learning Over Wireless Networks:
Principles and Applications [55.65768284748698]
Machine learning (ML) is a promising enabler for the fifth generation (5G) communication systems and beyond.
This article aims to provide a holistic overview of relevant communication and ML principles, and thereby present communication-efficient and distributed learning frameworks with selected use cases.
arXiv Detail & Related papers (2020-08-06T12:37:14Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.