Talk is Not Always Cheap: Promoting Wireless Sensing Models with Text Prompts

• URL: http://arxiv.org/abs/2504.14621v2
• Date: Tue, 22 Apr 2025 14:48:39 GMT
• Title: Talk is Not Always Cheap: Promoting Wireless Sensing Models with Text Prompts
• Authors: Zhenkui Yang, Zeyi Huang, Ge Wang, Han Ding, Tony Xiao Han, Fei Wang,
• Abstract summary: We propose an innovative text-enhanced wireless sensing framework, WiTalk, that seamlessly integrates semantic knowledge through three prompt strategies-label-only, brief description, and detailed action description.<n>We rigorously validate this framework across three public benchmark datasets: XRF55 for human action recognition (HAR), WiFiTAL and XRFV2 for WiFi temporal action localization.
• Score: 14.801020598640191
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Wireless signal-based human sensing technologies, such as WiFi, millimeter-wave (mmWave) radar, and Radio Frequency Identification (RFID), enable the detection and interpretation of human presence, posture, and activities, thereby providing critical support for applications in public security, healthcare, and smart environments. These technologies exhibit notable advantages due to their non-contact operation and environmental adaptability; however, existing systems often fail to leverage the textual information inherent in datasets. To address this, we propose an innovative text-enhanced wireless sensing framework, WiTalk, that seamlessly integrates semantic knowledge through three hierarchical prompt strategies-label-only, brief description, and detailed action description-without requiring architectural modifications or incurring additional data costs. We rigorously validate this framework across three public benchmark datasets: XRF55 for human action recognition (HAR), and WiFiTAL and XRFV2 for WiFi temporal action localization (TAL). Experimental results demonstrate significant performance improvements: on XRF55, accuracy for WiFi, RFID, and mmWave increases by 3.9%, 2.59%, and 0.46%, respectively; on WiFiTAL, the average performance of WiFiTAD improves by 4.98%; and on XRFV2, the mean average precision gains across various methods range from 4.02% to 13.68%. Our codes have been included in https://github.com/yangzhenkui/WiTalk.

Related papers

• Neuro-Symbolic Fusion of Wi-Fi Sensing Data for Passive Radar with Inter-Modal Knowledge Transfer [10.388561519507471]
  This paper introduces DeepProbHAR, a neuro-symbolic architecture for Wi-Fi sensing. It provides initial evidence that Wi-Fi signals can differentiate between simple movements, such as leg or arm movements. DeepProbHAR achieves results comparable to the state-of-the-art in human activity recognition.
  arXiv  Detail & Related papers  (2024-07-01T08:43:27Z)
• Physical-Layer Semantic-Aware Network for Zero-Shot Wireless Sensing [74.12670841657038]
  Device-free wireless sensing has recently attracted significant interest due to its potential to support a wide range of immersive human-machine interactive applications. Data heterogeneity in wireless signals and data privacy regulation of distributed sensing have been considered as the major challenges that hinder the wide applications of wireless sensing in large area networking systems. We propose a novel zero-shot wireless sensing solution that allows models constructed in one or a limited number of locations to be directly transferred to other locations without any labeled data.
  arXiv  Detail & Related papers  (2023-12-08T13:50:30Z)
• Joint Sensing, Communication, and AI: A Trifecta for Resilient THz User Experiences [118.91584633024907]
  A novel joint sensing, communication, and artificial intelligence (AI) framework is proposed so as to optimize extended reality (XR) experiences over terahertz (THz) wireless systems.
  arXiv  Detail & Related papers  (2023-04-29T00:39:50Z)
• Zero-Effort Two-Factor Authentication Using Wi-Fi Radio Wave Transmission and Machine Learning [0.0]
  This paper presents a novel zero-effort two-factor authentication (2FA) approach that combines the unique characteristics of a users environment and Machine Learning (ML) to confirm their identity. A prototype was developed using Raspberry Pi devices and experiments were conducted to demonstrate the effectiveness and practicality of the proposed approach. The proposed system holds great promise in revolutionizing the field of 2FA and user authentication, offering a new era of secure and seamless access to sensitive information.
  arXiv  Detail & Related papers  (2023-03-04T21:04:10Z)
• WiFi-based Spatiotemporal Human Action Perception [53.41825941088989]
  An end-to-end WiFi signal neural network (SNN) is proposed to enable WiFi-only sensing in both line-of-sight and non-line-of-sight scenarios. Especially, the 3D convolution module is able to explore thetemporal continuity of WiFi signals, and the feature self-attention module can explicitly maintain dominant features.
  arXiv  Detail & Related papers  (2022-06-20T16:03:45Z)
• Hands-on Wireless Sensing with Wi-Fi: A Tutorial [7.8774878397748065]
  This tutorial takes Wi-Fi sensing as an example. It introduces both the theoretical principles and the code implementation of data collection, signal processing, features extraction, and model design.
  arXiv  Detail & Related papers  (2022-06-20T01:53:35Z)
• A Wireless-Vision Dataset for Privacy Preserving Human Activity Recognition [53.41825941088989]
  A new WiFi-based and video-based neural network (WiNN) is proposed to improve the robustness of activity recognition. Our results show that WiVi data set satisfies the primary demand and all three branches in the proposed pipeline keep more than $80%$ of activity recognition accuracy.
  arXiv  Detail & Related papers  (2022-05-24T10:49:11Z)
• GraSens: A Gabor Residual Anti-aliasing Sensing Framework for Action Recognition using WiFi [52.530330427538885]
  WiFi-based human action recognition (HAR) has been regarded as a promising solution in applications such as smart living and remote monitoring. We propose an end-to-end Gabor residual anti-aliasing sensing network (GraSens) to directly recognize the actions using the WiFi signals from the wireless devices in diverse scenarios.
  arXiv  Detail & Related papers  (2022-05-24T10:20:16Z)
• RF-Net: a Unified Meta-learning Framework for RF-enabled One-shot Human Activity Recognition [9.135311655929366]
  Device-free (or contactless) sensing is more sensitive to environment changes than device-based (or wearable) sensing. Existing solutions to RF-HAR entail a laborious data collection process for adapting to new environments. We propose RF-Net as a meta-learning based approach to one-shot RF-HAR; it reduces the labeling efforts for environment adaptation to the minimum level.
  arXiv  Detail & Related papers  (2021-10-29T01:58:29Z)
• Vision Meets Wireless Positioning: Effective Person Re-identification with Recurrent Context Propagation [120.18969251405485]
  Existing person re-identification methods rely on the visual sensor to capture the pedestrians. Mobile phone can be sensed by WiFi and cellular networks in the form of a wireless positioning signal. We propose a novel recurrent context propagation module that enables information to propagate between visual data and wireless positioning data.
  arXiv  Detail & Related papers  (2020-08-10T14:19:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.