Integrating Semantic Communication and Human Decision-Making into an End-to-End Sensing-Decision Framework

• URL: http://arxiv.org/abs/2412.05103v1
• Date: Fri, 06 Dec 2024 15:01:19 GMT
• Title: Integrating Semantic Communication and Human Decision-Making into an End-to-End Sensing-Decision Framework
• Authors: Edgar Beck, Hsuan-Yu Lin, Patrick Rückert, Yongping Bao, Bettina von Helversen, Sebastian Fehrler, Kirsten Tracht, Armin Dekorsy,
• Abstract summary: We propose to integrate semantic communication and Human Decision-Making (HDM) into one probabilistic end-to-end sensing-decision framework.<n>Our study provides new insights for the design/interaction of semantic communication with models of HDM.
• Score: 5.64132780952967
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: As early as 1949, Weaver defined communication in a very broad sense to include all procedures by which one mind or technical system can influence another, thus establishing the idea of semantic communication. With the recent success of machine learning in expert assistance systems where sensed information is wirelessly provided to a human to assist task execution, the need to design effective and efficient communications has become increasingly apparent. In particular, semantic communication aims to convey the meaning behind the sensed information relevant for Human Decision-Making (HDM). Regarding the interplay between semantic communication and HDM, many questions remain, such as how to model the entire end-to-end sensing-decision-making process, how to design semantic communication for the HDM and which information should be provided to the HDM. To address these questions, we propose to integrate semantic communication and HDM into one probabilistic end-to-end sensing-decision framework that bridges communications and psychology. In our interdisciplinary framework, we model the human through a HDM process, allowing us to explore how feature extraction from semantic communication can best support human decision-making. In this sense, our study provides new insights for the design/interaction of semantic communication with models of HDM. Our initial analysis shows how semantic communication can balance the level of detail with human cognitive capabilities while demanding less bandwidth, power, and latency.

Related papers

• Latent Space Alignment for AI-Native MIMO Semantic Communications [11.185200213307208]
  We propose a novel approach to address semantic mismatches in semantic communications.<n>A neural network-based model learns semantic precoder/decoder under power budget and complexity constraints.<n>Results demonstrate the effectiveness of the proposed approach in a goal-oriented semantic communication scenario.
  arXiv  Detail & Related papers  (2025-07-22T15:16:18Z)
• A Human Digital Twin Architecture for Knowledge-based Interactions and Context-Aware Conversations [0.9580312063277943]
  Recent developments in Artificial Intelligence (AI) and Machine Learning (ML) are creating new opportunities for Human-Autonomy Teaming (HAT) We present a real-time Human Digital Twin (HDT) architecture that integrates Large Language Models (LLMs) for knowledge reporting, answering, and recommendation, embodied in a visual interface. The HDT acts as a visually and behaviorally realistic team member, integrated throughout the mission lifecycle, from training to deployment to after-action review.
  arXiv  Detail & Related papers  (2025-04-04T03:56:26Z)
• Multi-Modal Self-Supervised Semantic Communication [52.76990720898666]
  We propose a multi-modal semantic communication system that leverages multi-modal self-supervised learning to enhance task-agnostic feature extraction. The proposed approach effectively captures both modality-invariant and modality-specific features while minimizing training-related communication overhead. The findings underscore the advantages of multi-modal self-supervised learning in semantic communication, paving the way for more efficient and scalable edge inference systems.
  arXiv  Detail & Related papers  (2025-03-18T06:13:02Z)
• Multimodal Fusion with LLMs for Engagement Prediction in Natural Conversation [70.52558242336988]
  We focus on predicting engagement in dyadic interactions by scrutinizing verbal and non-verbal cues, aiming to detect signs of disinterest or confusion. In this work, we collect a dataset featuring 34 participants engaged in casual dyadic conversations, each providing self-reported engagement ratings at the end of each conversation. We introduce a novel fusion strategy using Large Language Models (LLMs) to integrate multiple behavior modalities into a multimodal transcript''
  arXiv  Detail & Related papers  (2024-09-13T18:28:12Z)
• All rivers run into the sea: Unified Modality Brain-like Emotional Central Mechanism [32.742064026327334]
  We propose UMBEnet, a brain-like unified modal affective processing network. The primary design of UMBEnet includes a Dual-Stream (DS) structure that fuses inherent prompts with a Prompt Pool and a Sparse Feature Fusion (SFF) module. In experiments on the largest benchmark datasets in the Dynamic Facial Expression Recognition (DFER) field, UMBEnet consistently outperforms the current state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-07-22T12:26:31Z)
• PersLLM: A Personified Training Approach for Large Language Models [66.16513246245401]
  We propose PersLLM, a framework for better data construction and model tuning.<n>For insufficient data usage, we incorporate strategies such as Chain-of-Thought prompting and anti-induction.<n>For rigid behavior patterns, we design the tuning process and introduce automated DPO to enhance the specificity and dynamism of the models' personalities.
  arXiv  Detail & Related papers  (2024-07-17T08:13:22Z)
• Semantic Communication Networks Empowered Artificial Intelligence of Things [2.590720801978138]
  This paper presents a comprehensive survey of security and privacy threats across various layers of semantic communication systems. We identify critical open issues in this burgeoning field warranting further investigation.
  arXiv  Detail & Related papers  (2024-07-04T14:39:28Z)
• Reasoning with the Theory of Mind for Pragmatic Semantic Communication [62.87895431431273]
  A pragmatic semantic communication framework is proposed in this paper. It enables effective goal-oriented information sharing between two-intelligent agents. Numerical evaluations demonstrate the framework's ability to achieve efficient communication with a reduced amount of bits.
  arXiv  Detail & Related papers  (2023-11-30T03:36:19Z)
• Reasoning over the Air: A Reasoning-based Implicit Semantic-Aware Communication Framework [124.6509194665514]
  A novel implicit semantic-aware communication (iSAC) architecture is proposed for representing, communicating, and interpreting the implicit semantic meaning between source and destination users. A projection-based semantic encoder is proposed to convert the high-dimensional graphical representation of explicit semantics into a low-dimensional semantic constellation space for efficient physical channel transmission. A generative adversarial imitation learning-based solution, called G-RML, is proposed to enable the destination user to learn and imitate the implicit semantic reasoning process of source user.
  arXiv  Detail & Related papers  (2023-06-20T01:32:27Z)
• Less Data, More Knowledge: Building Next Generation Semantic Communication Networks [180.82142885410238]
  We present the first rigorous vision of a scalable end-to-end semantic communication network. We first discuss how the design of semantic communication networks requires a move from data-driven networks towards knowledge-driven ones. By using semantic representation and languages, we show that the traditional transmitter and receiver now become a teacher and apprentice.
  arXiv  Detail & Related papers  (2022-11-25T19:03:25Z)
• Emergent Quantized Communication [34.31732248872158]
  We propose an alternative approach to achieve discrete communication -- quantization of communicated messages. Message quantization allows us to train the model end-to-end, achieving superior performance in multiple setups.
  arXiv  Detail & Related papers  (2022-11-04T12:39:45Z)
• Imitation Learning-based Implicit Semantic-aware Communication Networks: Multi-layer Representation and Collaborative Reasoning [68.63380306259742]
  Despite its promising potential, semantic communications and semantic-aware networking are still at their infancy. We propose a novel reasoning-based implicit semantic-aware communication network architecture that allows multiple tiers of CDC and edge servers to collaborate. We introduce a new multi-layer representation of semantic information taking into consideration both the hierarchical structure of implicit semantics as well as the personalized inference preference of individual users.
  arXiv  Detail & Related papers  (2022-10-28T13:26:08Z)
• Towards Human-Agent Communication via the Information Bottleneck Principle [19.121541894577298]
  We study how trading off these three factors -- utility, informativeness, and complexity -- shapes emergent communication. We propose Vector-Quantized Variational Information Bottleneck (VQ-VIB), a method for training neural agents to compress inputs into discrete signals embedded in a continuous space.
  arXiv  Detail & Related papers  (2022-06-30T20:10:20Z)
• Neuro-Symbolic Artificial Intelligence (AI) for Intent based Semantic Communication [85.06664206117088]
  6G networks must consider semantics and effectiveness (at end-user) of the data transmission. NeSy AI is proposed as a pillar for learning causal structure behind the observed data. GFlowNet is leveraged for the first time in a wireless system to learn the probabilistic structure which generates the data.
  arXiv  Detail & Related papers  (2022-05-22T07:11:57Z)
• Semantic Information Recovery in Wireless Networks [8.508198765617195]
  We present an ML-based semantic communication system SINFONY. SINFONY communicates the meaning behind multiple messages to a single receiver for semantic recovery. Numerical results reveal a tremendous rate-normalized SNR shift up to 20 dB compared to classically designed communication systems.
  arXiv  Detail & Related papers  (2022-04-28T09:17:50Z)
• You Impress Me: Dialogue Generation via Mutual Persona Perception [62.89449096369027]
  The research in cognitive science suggests that understanding is an essential signal for a high-quality chit-chat conversation. Motivated by this, we propose P2 Bot, a transmitter-receiver based framework with the aim of explicitly modeling understanding.
  arXiv  Detail & Related papers  (2020-04-11T12:51:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.