Attention to Entropic Communication

• URL: http://arxiv.org/abs/2307.11423v2
• Date: Tue, 9 Jan 2024 17:31:20 GMT
• Title: Attention to Entropic Communication
• Authors: Torsten En{\ss}lin, Carolin Weidinger, Philipp Frank
• Abstract summary: Relative entropy (RE), aka Kullback-Leibler divergence, plays a central role in communication theory. We combine these concepts, attention and RE. RE guides optimal encoding of messages in bandwidth-limited communication.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The concept of attention, numerical weights that emphasize the importance of particular data, has proven to be very relevant in artificial intelligence. Relative entropy (RE, aka Kullback-Leibler divergence) plays a central role in communication theory. Here we combine these concepts, attention and RE. RE guides optimal encoding of messages in bandwidth-limited communication as well as optimal message decoding via the maximum entropy principle (MEP). In the coding scenario, RE can be derived from four requirements, namely being analytical, local, proper, and calibrated. Weighted RE, used for attention steering in communications, turns out to be improper. To see how proper attention communication can emerge, we analyze a scenario of a message sender who wants to ensure that the receiver of the message can perform well-informed actions. If the receiver decodes the message using the MEP, the sender only needs to know the receiver's utility function to inform optimally, but not the receiver's initial knowledge state. In case only the curvature of the utility function maxima are known, it becomes desirable to accurately communicate an attention function, in this case a by this curvature weighted and re-normalized probability function. Entropic attention communication is here proposed as the desired generalization of entropic communication that permits weighting while being proper, thereby aiding the design of optimal communication protocols in technical applications and helping to understand human communication. For example, our analysis shows how to derive the level of cooperation expected under misaligned interests of otherwise honest communication partners.

Related papers

• A Memory-Based Reinforcement Learning Approach to Integrated Sensing and Communication [52.40430937325323]
  We consider a point-to-point integrated sensing and communication (ISAC) system, where a transmitter conveys a message to a receiver over a channel with memory. We formulate the capacity-distortion tradeoff for the ISAC problem when sensing is performed in an online fashion.
  arXiv  Detail & Related papers  (2024-12-02T03:30:50Z)
• Unbounded quantum advantage in communication complexity measured by distinguishability [0.0]
  We measure the complexity of a task by the minimal distinguishability required to accomplish it. We show that the classical-to-quantum ratio of minimal distinguishability required to achieve the same success metric escalates exponentially.
  arXiv  Detail & Related papers  (2024-01-23T16:48:59Z)
• Rate-Distortion-Perception Theory for Semantic Communication [73.04341519955223]
  We study the achievable data rate of semantic communication under the symbol distortion and semantic perception constraints. We observe that there exists cases that the receiver can directly infer the semantic information source satisfying certain distortion and perception constraints.
  arXiv  Detail & Related papers  (2023-12-09T02:04:32Z)
• Reasoning with the Theory of Mind for Pragmatic Semantic Communication [62.87895431431273]
  A pragmatic semantic communication framework is proposed in this paper. It enables effective goal-oriented information sharing between two-intelligent agents. Numerical evaluations demonstrate the framework's ability to achieve efficient communication with a reduced amount of bits.
  arXiv  Detail & Related papers  (2023-11-30T03:36:19Z)
• Reasoning over the Air: A Reasoning-based Implicit Semantic-Aware Communication Framework [124.6509194665514]
  A novel implicit semantic-aware communication (iSAC) architecture is proposed for representing, communicating, and interpreting the implicit semantic meaning between source and destination users. A projection-based semantic encoder is proposed to convert the high-dimensional graphical representation of explicit semantics into a low-dimensional semantic constellation space for efficient physical channel transmission. A generative adversarial imitation learning-based solution, called G-RML, is proposed to enable the destination user to learn and imitate the implicit semantic reasoning process of source user.
  arXiv  Detail & Related papers  (2023-06-20T01:32:27Z)
• Semantic Communication of Learnable Concepts [16.373044313375782]
  We consider the problem of communicating a sequence of concepts, i.e., unknown and potentially maps, which can be observed only through examples. The transmitter applies a learning algorithm to the available examples, and extracts knowledge from the data. The transmitter then needs to communicate the learned models to a remote receiver through a rate-limited channel.
  arXiv  Detail & Related papers  (2023-05-14T11:16:17Z)
• Knowledge Enhanced Semantic Communication Receiver [7.171974845607281]
  We propose a knowledge enhanced semantic communication framework in which the receiver can more actively utilize the facts in the knowledge base for semantic reasoning and decoding. Specifically, we design a transformer-based knowledge extractor to find relevant factual triples for the received noisy signal. Extensive simulation results on the WebNLG dataset demonstrate that the proposed receiver yields superior performance on top of the knowledge graph enhanced decoding.
  arXiv  Detail & Related papers  (2023-02-13T01:49:51Z)
• Adversarial Learning for Implicit Semantic-Aware Communications [104.08383219177557]
  We develop a novel adversarial learning-based implicit semantic-aware communication architecture (iSAC) We prove that by applying iSAC, the destination user can always learn an inference rule that matches the true inference rule of the source messages. Experimental results show that the proposed iSAC can offer up to a 19.69 dB improvement over existing non-inferential communication solutions.
  arXiv  Detail & Related papers  (2023-01-27T08:28:12Z)
• Neuro-Symbolic Causal Reasoning Meets Signaling Game for Emergent Semantic Communications [71.63189900803623]
  A novel emergent SC system framework is proposed and is composed of a signaling game for emergent language design and a neuro-symbolic (NeSy) artificial intelligence (AI) approach for causal reasoning. The ESC system is designed to enhance the novel metrics of semantic information, reliability, distortion and similarity.
  arXiv  Detail & Related papers  (2022-10-21T15:33:37Z)
• Adaptive advantage in entanglement-assisted communications [0.0]
  Entanglement-assisted classical communication protocols usually consist of two successive rounds. We show that adaptive protocols improve the success probability in Random Access Codes. We briefly discuss extension of these ideas to scenarios involving quantum communication.
  arXiv  Detail & Related papers  (2022-03-10T13:54:02Z)
• Learning Semantics: An Opportunity for Effective 6G Communications [8.262718096663077]
  semantic communications are envisioned as a key enabler of future 6G networks. This work explores the opportunity offered by semantic communications for beyond 5G networks. We present and detail a novel architecture that enables representation learning of semantic symbols for effective semantic communications.
  arXiv  Detail & Related papers  (2021-10-14T08:00:54Z)
• Quasi-Equivalence Discovery for Zero-Shot Emergent Communication [63.175848843466845]
  We present a novel problem setting and the Quasi-Equivalence Discovery algorithm that allows for zero-shot coordination (ZSC) We show that these two factors lead to unique optimal ZSC policies in referential games. QED can iteratively discover the symmetries in this setting and converges to the optimal ZSC policy.
  arXiv  Detail & Related papers  (2021-03-14T23:42:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.