Dual-Attention and Adversarial Transfer Networks for Sim-to-Real Cross-Orientation Wireless Sensing

• URL: http://arxiv.org/abs/2608.05664v1
• Date: Thu, 06 Aug 2026 07:00:48 GMT
• Title: Dual-Attention and Adversarial Transfer Networks for Sim-to-Real Cross-Orientation Wireless Sensing
• Abstract summary: We develop a physics-guided simulator that synthesizes orientation-diverse wireless training data from single-orientation motion.<n>We also propose a dual-attention network that extracts activity-discriminative and orientation-robust representations from dual-link Doppler spectrograms.<n>S2M-Sense achieves 88.33% recognition accuracy using only the dual-link multi-orientation simulated dataset.
• Score: 12.205084145418859
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Millimeter-wave human activity recognition suffers significant performance degradation when the user's orientation changes relative to the sensing system, yet collecting labeled multi-orientation data is labor-intensive and costly. To eliminate the need for exhaustive multi-orientation measured data, we develop a physics-guided simulator that synthesizes orientation-diverse wireless training data from single-orientation motion. Specifically, to suppress orientation-induced feature variations, we propose a dual-attention network that extracts activity-discriminative and orientation-robust representations from dual-link Doppler spectrograms. To bridge the simulation-to-reality gap, we introduce an adversarial unsupervised transfer learning mechanism that aligns feature distributions using only a small number of unlabeled target-domain samples. The S2M-Sense platform shows high fidelity in reproducing real-world signatures, validated against 60.48 GHz mmWave measured data with an average structural similarity index measure (SSIM) of 0.84 between simulated and measured Doppler spectrograms across all 4 activities and 4 orientations. Experimental results show that S2M-Sense achieves 88.33% recognition accuracy using only the dual-link multi-orientation simulated dataset, which improves to 95% after simulation-to-reality transfer learning with as few as 16 unlabeled measured samples. Both cases with and without transfer learning outperform state-of-the-art cross-domain sensing methods.

Related papers

• Mind the Rift: Cross-Scale Coupling Mismatch for AI-Generated Video Detection [10.264738258468254]
  Cross-scale coupling mismatch is a new forensic signal.<n>In natural videos, macro-level temporal dynamics and micro-level residual patterns are intrinsically coupled by the unified imaging physics pipeline.<n>But AI generators, whose training objectives do not explicitly preserve this joint distribution, systematically violate this coupling.<n>Detecting such mismatch is challenging because it requires independently extracting information at both scales.
  arXiv  Detail & Related papers  (2026-09-01T05:23:48Z)
• WiFuse: An Attention Mechanism for Human Activity Recognition using Fused CSI Amplitude and Delay-Doppler Channel Features [1.0554048699217669]
  WiFuse is a dual-stream Channel State Information framework for human activity recognition.<n>It pairs amplitude and Delay-Doppler representations within a dual-stream strategy, enhanced by transfer learning.<n>It reaches an overall accuracy of up to 95.28% across the four environments of the XRF55 dataset and up to 98.20% on the multi-user Wi-MIR dataset.
  arXiv  Detail & Related papers  (2026-08-01T12:54:09Z)
• TactSpace: Learning a Physics-enriched Shared Latent Space for Tactile Sim-to-Real Transfer [8.357674069719524]
  We propose a multi-modal representation learning framework that aligns heterogeneous tactile modalities within a shared latent space.<n>Our approach employs modality-specific encoders to project diverse tactile observations into a common embedding space.<n>Our results demonstrate zero-shot sim-to-real transfer across physically dissimilar representations.
  arXiv  Detail & Related papers  (2026-06-17T11:41:27Z)
• BabyMamba-HAR: Lightweight Selective State Space Models for Efficient Human Activity Recognition on Resource Constrained Devices [0.0]
  Human activity recognition (HAR) on wearable and mobile devices is constrained by memory footprint and computational budget.<n> Selective state space models (SSMs) offer linear time sequence processing with input dependent gating.<n>BabyMamba-HAR is introduced, a framework comprising two novel lightweight Mamba inspired architectures for resource constrained HAR.
  arXiv  Detail & Related papers  (2026-02-10T15:16:32Z)
• Flexible Gravitational-Wave Parameter Estimation with Transformers [73.44614054040267]
  We introduce a flexible transformer-based architecture paired with a training strategy that enables adaptation to diverse analysis settings at inference time.<n>We demonstrate that a single flexible model -- called Dingo-T1 -- can analyze 48 gravitational-wave events from the third LIGO-Virgo-KAGRA Observing Run.
  arXiv  Detail & Related papers  (2025-12-02T17:49:08Z)
• Domain Adaptive Synapse Detection with Weak Point Annotations [63.97144211520869]
  We present AdaSyn, a framework for domain adaptive synapse detection with weak point annotations. In the WASPSYN challenge at I SBI 2023, our method ranks the 1st place.
  arXiv  Detail & Related papers  (2023-08-31T05:05:53Z)
• S2R-ViT for Multi-Agent Cooperative Perception: Bridging the Gap from Simulation to Reality [41.25312194294171]
  We propose the first-to-Reality transfer learning framework for multi-agent cooperative perception using a novel Vision Transformer, named as S2R-ViT. Our experiments on the public multi-agent cooperative perception datasets OPV2V and V2V4Real demonstrate that the proposed S2R-ViT can effectively bridge the gap from simulation to reality.
  arXiv  Detail & Related papers  (2023-07-16T03:54:10Z)
• Joint Spatial-Temporal and Appearance Modeling with Transformer for Multiple Object Tracking [59.79252390626194]
  We propose a novel solution named TransSTAM, which leverages Transformer to model both the appearance features of each object and the spatial-temporal relationships among objects. The proposed method is evaluated on multiple public benchmarks including MOT16, MOT17, and MOT20, and it achieves a clear performance improvement in both IDF1 and HOTA.
  arXiv  Detail & Related papers  (2022-05-31T01:19:18Z)
• Towards Scale Consistent Monocular Visual Odometry by Learning from the Virtual World [83.36195426897768]
  We propose VRVO, a novel framework for retrieving the absolute scale from virtual data. We first train a scale-aware disparity network using both monocular real images and stereo virtual data. The resulting scale-consistent disparities are then integrated with a direct VO system.
  arXiv  Detail & Related papers  (2022-03-11T01:51:54Z)
• Vertical Federated Edge Learning with Distributed Integrated Sensing and Communication [40.84033154889936]
  This letter studies a vertical federated edge learning (FEEL) system for collaborative objects/human motion recognition. In this system, distributed edge devices first send wireless signals to sense targeted objects/human, and then exchange intermediate computed vectors for collaborative recognition. By considering a human motion recognition task, experimental results show that our vertical FEEL based approach achieves recognition accuracy up to 98% with an improvement up to 8% compared to the benchmarks.
  arXiv  Detail & Related papers  (2022-01-21T02:05:07Z)
• Domain Adaptive Robotic Gesture Recognition with Unsupervised Kinematic-Visual Data Alignment [60.31418655784291]
  We propose a novel unsupervised domain adaptation framework which can simultaneously transfer multi-modality knowledge, i.e., both kinematic and visual data, from simulator to real robot. It remedies the domain gap with enhanced transferable features by using temporal cues in videos, and inherent correlations in multi-modal towards recognizing gesture. Results show that our approach recovers the performance with great improvement gains, up to 12.91% in ACC and 20.16% in F1score without using any annotations in real robot.
  arXiv  Detail & Related papers  (2021-03-06T09:10:03Z)
• MetricUNet: Synergistic Image- and Voxel-Level Learning for Precise CT Prostate Segmentation via Online Sampling [66.01558025094333]
  We propose a two-stage framework, with the first stage to quickly localize the prostate region and the second stage to precisely segment the prostate. We introduce a novel online metric learning module through voxel-wise sampling in the multi-task network. Our method can effectively learn more representative voxel-level features compared with the conventional learning methods with cross-entropy or Dice loss.
  arXiv  Detail & Related papers  (2020-05-15T10:37:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.