Radio Frequency Signal based Human Silhouette Segmentation: A Sequential Diffusion Approach

• URL: http://arxiv.org/abs/2407.19244v1
• Date: Sat, 27 Jul 2024 12:44:21 GMT
• Title: Radio Frequency Signal based Human Silhouette Segmentation: A Sequential Diffusion Approach
• Authors: Penghui Wen, Kun Hu, Dong Yuan, Zhiyuan Ning, Changyang Li, Zhiyong Wang,
• Abstract summary: We propose a two-stage Sequential Diffusion Model (SDM) to synthesize high-quality segmentation jointly. Cross-view blocks are devised to guide the diffusion model in a multi-scale manner. temporal blocks are devised to fine-tune the frame-level model to incorporate frequency-temporal contexts and motion dynamics.
• Score: 26.987963024941635
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Radio frequency (RF) signals have been proved to be flexible for human silhouette segmentation (HSS) under complex environments. Existing studies are mainly based on a one-shot approach, which lacks a coherent projection ability from the RF domain. Additionally, the spatio-temporal patterns have not been fully explored for human motion dynamics in HSS. Therefore, we propose a two-stage Sequential Diffusion Model (SDM) to progressively synthesize high-quality segmentation jointly with the considerations on motion dynamics. Cross-view transformation blocks are devised to guide the diffusion model in a multi-scale manner for comprehensively characterizing human related patterns in an individual frame such as directional projection from signal planes. Moreover, spatio-temporal blocks are devised to fine-tune the frame-level model to incorporate spatio-temporal contexts and motion dynamics, enhancing the consistency of the segmentation maps. Comprehensive experiments on a public benchmark -- HIBER demonstrate the state-of-the-art performance of our method with an IoU 0.732. Our code is available at https://github.com/ph-w2000/SDM.

Related papers

• Electromyography-Based Gesture Recognition: Hierarchical Feature Extraction for Enhanced Spatial-Temporal Dynamics [0.7083699704958353]
  We propose a lightweight squeeze-excitation deep learning-based multi stream spatial temporal dynamics time-varying feature extraction approach. The proposed model was tested on the Ninapro DB2, DB4, and DB5 datasets, achieving accuracy rates of 96.41%, 92.40%, and 93.34%, respectively.
  arXiv  Detail & Related papers  (2025-04-04T07:11:12Z)
• GestureLSM: Latent Shortcut based Co-Speech Gesture Generation with Spatial-Temporal Modeling [32.47567372398872]
  GestureLSM is a flow-matching-based approach for Co-Speech Gesture Generation with spatial-temporal modeling. It achieves state-of-the-art performance on BEAT2 while significantly reducing inference time compared to existing methods.
  arXiv  Detail & Related papers  (2025-01-31T05:34:59Z)
• MHSA: A Multi-scale Hypergraph Network for Mild Cognitive Impairment Detection via Synchronous and Attentive Fusion [4.526574526136158]
  A Multi-scale Hypergraph Network for MCI Detection via Synchronous and Attentive Fusion is presented. Our approach employs the Phase-Locking Value (PLV) to calculate the phase synchronization relationship in the spectrum domain of regions of interest. We structure the PLV coefficients dynamically adjust strategy, and the dynamic hypergraph is modelled based on a comprehensive temporal-spectrum fusion matrix.
  arXiv  Detail & Related papers  (2024-12-11T02:59:57Z)
• Multi-Resolution Generative Modeling of Human Motion from Limited Data [3.5229503563299915]
  We present a generative model that learns to synthesize human motion from limited training sequences. The model adeptly captures human motion patterns by integrating skeletal convolution layers and a multi-scale architecture.
  arXiv  Detail & Related papers  (2024-11-25T15:36:29Z)
• KFD-NeRF: Rethinking Dynamic NeRF with Kalman Filter [49.85369344101118]
  We introduce KFD-NeRF, a novel dynamic neural radiance field integrated with an efficient and high-quality motion reconstruction framework based on Kalman filtering. Our key idea is to model the dynamic radiance field as a dynamic system whose temporally varying states are estimated based on two sources of knowledge: observations and predictions. Our KFD-NeRF demonstrates similar or even superior performance within comparable computational time and state-of-the-art view synthesis performance with thorough training.
  arXiv  Detail & Related papers  (2024-07-18T05:48:24Z)
• Synthetic location trajectory generation using categorical diffusion models [50.809683239937584]
  Diffusion models (DPMs) have rapidly evolved to be one of the predominant generative models for the simulation of synthetic data. We propose using DPMs for the generation of synthetic individual location trajectories (ILTs) which are sequences of variables representing physical locations visited by individuals.
  arXiv  Detail & Related papers  (2024-02-19T15:57:39Z)
• Towards Detailed Text-to-Motion Synthesis via Basic-to-Advanced Hierarchical Diffusion Model [60.27825196999742]
  We propose a novel Basic-to-Advanced Hierarchical Diffusion Model, named B2A-HDM, to collaboratively exploit low-dimensional and high-dimensional diffusion models for detailed motion synthesis. Specifically, the basic diffusion model in low-dimensional latent space provides the intermediate denoising result that is consistent with the textual description. The advanced diffusion model in high-dimensional latent space focuses on the following detail-enhancing denoising process.
  arXiv  Detail & Related papers  (2023-12-18T06:30:39Z)
• A Decoupled Spatio-Temporal Framework for Skeleton-based Action Segmentation [89.86345494602642]
  Existing methods are limited in weak-temporal modeling capability. We propose a Decoupled Scoupled Framework (DeST) to address the issues. DeST significantly outperforms current state-of-the-art methods with less computational complexity.
  arXiv  Detail & Related papers  (2023-12-10T09:11:39Z)
• Generative Modeling with Phase Stochastic Bridges [49.4474628881673]
  Diffusion models (DMs) represent state-of-the-art generative models for continuous inputs. We introduce a novel generative modeling framework grounded in textbfphase space dynamics Our framework demonstrates the capability to generate realistic data points at an early stage of dynamics propagation.
  arXiv  Detail & Related papers  (2023-10-11T18:38:28Z)
• Adversarial Training of Denoising Diffusion Model Using Dual Discriminators for High-Fidelity Multi-Speaker TTS [0.0]
  The diffusion model is capable of generating high-quality data through a probabilistic approach. It suffers from the drawback of slow generation speed due to the requirement of a large number of time steps. We propose a speech synthesis model with two discriminators: a diffusion discriminator for learning the distribution of the reverse process and a spectrogram discriminator for learning the distribution of the generated data.
  arXiv  Detail & Related papers  (2023-08-03T07:22:04Z)
• Data-driven modelling of brain activity using neural networks, Diffusion Maps, and the Koopman operator [0.0]
  We propose a machine-learning approach to model long-term out-of-sample dynamics of brain activity from task-dependent fMRI data. We use Diffusion maps (DMs) to discover a set of variables that parametrize the low-dimensional manifold on which the emergent high-dimensional fMRI time series evolve. We construct reduced-order-models (ROMs) on the embedded manifold via two techniques: Feedforward Neural Networks (FNNs) and the Koopman operator.
  arXiv  Detail & Related papers  (2023-04-24T09:08:12Z)
• Modiff: Action-Conditioned 3D Motion Generation with Denoising Diffusion Probabilistic Models [58.357180353368896]
  We propose a conditional paradigm that benefits from the denoising diffusion probabilistic model (DDPM) to tackle the problem of realistic and diverse action-conditioned 3D skeleton-based motion generation. We are a pioneering attempt that uses DDPM to synthesize a variable number of motion sequences conditioned on a categorical action.
  arXiv  Detail & Related papers  (2023-01-10T13:15:42Z)
• JNMR: Joint Non-linear Motion Regression for Video Frame Interpolation [47.123769305867775]
  Video frame (VFI) aims to generate frames by warping learnable motions from the bidirectional historical references. We reformulate VFI as a Joint Non-linear Motion Regression (JNMR) strategy to model the complicated motions of inter-frame. We show that the effectiveness and significant improvement of joint motion regression compared with state-of-the-art methods.
  arXiv  Detail & Related papers  (2022-06-09T02:47:29Z)
• TSI: Temporal Saliency Integration for Video Action Recognition [32.18535820790586]
  We propose a Temporal Saliency Integration (TSI) block, which mainly contains a Salient Motion Excitation (SME) module and a Cross-scale Temporal Integration (CTI) module. SME aims to highlight the motion-sensitive area through local-global motion modeling. CTI is designed to perform multi-scale temporal modeling through a group of separate 1D convolutions respectively.
  arXiv  Detail & Related papers  (2021-06-02T11:43:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.