MultiFormer: A Multi-Person Pose Estimation System Based on CSI and Attention Mechanism

• URL: http://arxiv.org/abs/2505.22555v1
• Date: Wed, 28 May 2025 16:36:02 GMT
• Title: MultiFormer: A Multi-Person Pose Estimation System Based on CSI and Attention Mechanism
• Authors: Yanyi Qu, Haoyang Ma, Wenhui Xiong,
• Abstract summary: Human pose estimation based on Channel State Information (CSI) has emerged as a promising approach for non-intrusive and precise human activity monitoring.<n>This paper presents MultiFormer, a wireless sensing system that accurately estimates human pose through CSI.
• Score: 0.7373617024876725
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Human pose estimation based on Channel State Information (CSI) has emerged as a promising approach for non-intrusive and precise human activity monitoring, yet faces challenges including accurate multi-person pose recognition and effective CSI feature learning. This paper presents MultiFormer, a wireless sensing system that accurately estimates human pose through CSI. The proposed system adopts a Transformer based time-frequency dual-token feature extractor with multi-head self-attention. This feature extractor is able to model inter-subcarrier correlations and temporal dependencies of the CSI. The extracted CSI features and the pose probability heatmaps are then fused by Multi-Stage Feature Fusion Network (MSFN) to enforce the anatomical constraints. Extensive experiments conducted on on the public MM-Fi dataset and our self-collected dataset show that the MultiFormer achieves higher accuracy over state-of-the-art approaches, especially for high-mobility keypoints (wrists, elbows) that are particularly difficult for previous methods to accurately estimate.

Related papers

• Optimizing Multispectral Object Detection: A Bag of Tricks and Comprehensive Benchmarks [49.84182981950623]
  Multispectral object detection, utilizing RGB and TIR (thermal infrared) modalities, is widely recognized as a challenging task.<n>It requires not only the effective extraction of features from both modalities and robust fusion strategies, but also the ability to address issues such as spectral discrepancies.<n>We introduce an efficient and easily deployable multispectral object detection framework that can seamlessly optimize high-performing single-modality models.
  arXiv  Detail & Related papers  (2024-11-27T12:18:39Z)
• Transforming Multidimensional Time Series into Interpretable Event Sequences for Advanced Data Mining [5.2863523790908955]
  This paper introduces a novel proposed representation model designed to address the limitations of traditional methods in multidimensional time series (MTS) analysis. The proposed framework has significant potential for applications across various fields, including services for monitoring and optimizing IT infrastructure, medical diagnosis through continuous patient monitoring, trend analysis, and internet businesses for tracking user behavior and forecasting.
  arXiv  Detail & Related papers  (2024-09-22T06:27:07Z)
• Advancing Automated Deception Detection: A Multimodal Approach to Feature Extraction and Analysis [0.0]
  This research focuses on the extraction and combination of various features to enhance the accuracy of deception detection models. By systematically extracting features from visual, audio, and text data, and experimenting with different combinations, we developed a robust model that achieved an impressive 99% accuracy.
  arXiv  Detail & Related papers  (2024-07-08T14:59:10Z)
• Opinion-Unaware Blind Image Quality Assessment using Multi-Scale Deep Feature Statistics [54.08757792080732]
  We propose integrating deep features from pre-trained visual models with a statistical analysis model to achieve opinion-unaware BIQA (OU-BIQA) Our proposed model exhibits superior consistency with human visual perception compared to state-of-the-art BIQA models.
  arXiv  Detail & Related papers  (2024-05-29T06:09:34Z)
• Enhancing Apparent Personality Trait Analysis with Cross-Modal Embeddings [0.5461938536945723]
  We present a multimodal deep neural network with a Siamese extension for apparent personality trait prediction trained on short video recordings. Due to the highly centralized target distribution of the analyzed dataset, the changes in the third digit are relevant.
  arXiv  Detail & Related papers  (2024-05-06T20:51:28Z)
• MMA-DFER: MultiModal Adaptation of unimodal models for Dynamic Facial Expression Recognition in-the-wild [81.32127423981426]
  Multimodal emotion recognition based on audio and video data is important for real-world applications. Recent methods have focused on exploiting advances of self-supervised learning (SSL) for pre-training of strong multimodal encoders. We propose a different perspective on the problem and investigate the advancement of multimodal DFER performance by adapting SSL-pre-trained disjoint unimodal encoders.
  arXiv  Detail & Related papers  (2024-04-13T13:39:26Z)
• Exploiting Modality-Specific Features For Multi-Modal Manipulation Detection And Grounding [54.49214267905562]
  We construct a transformer-based framework for multi-modal manipulation detection and grounding tasks. Our framework simultaneously explores modality-specific features while preserving the capability for multi-modal alignment. We propose an implicit manipulation query (IMQ) that adaptively aggregates global contextual cues within each modality.
  arXiv  Detail & Related papers  (2023-09-22T06:55:41Z)
• MMNet: Multi-Collaboration and Multi-Supervision Network for Sequential Deepfake Detection [81.59191603867586]
  Sequential deepfake detection aims to identify forged facial regions with the correct sequence for recovery. The recovery of forged images requires knowledge of the manipulation model to implement inverse transformations. We propose Multi-Collaboration and Multi-Supervision Network (MMNet) that handles various spatial scales and sequential permutations in forged face images.
  arXiv  Detail & Related papers  (2023-07-06T02:32:08Z)
• Time-Selective RNN for Device-Free Multi-Room Human Presence Detection Using WiFi CSI [9.927073290898848]
  Device-free human presence detection is crucial technology for various applications, including home automation, security, and healthcare. Recent research has explored the use of wireless channel state information extracted from commercial WiFi access points (APs) to provide detailed channel characteristics. We propose a device-free human presence detection system for multi-room scenarios using a time-selective conditional dual feature extract recurrent network.
  arXiv  Detail & Related papers  (2023-04-25T19:21:47Z)
• Multimodal Adaptive Fusion of Face and Gait Features using Keyless attention based Deep Neural Networks for Human Identification [67.64124512185087]
  Soft biometrics such as gait are widely used with face in surveillance tasks like person recognition and re-identification. We propose a novel adaptive multi-biometric fusion strategy for the dynamic incorporation of gait and face biometric cues by leveraging keyless attention deep neural networks.
  arXiv  Detail & Related papers  (2023-03-24T05:28:35Z)
• In-Bed Human Pose Estimation from Unseen and Privacy-Preserving Image Domains [22.92165116962952]
  In-bed human posture estimation provides important health-related metrics with potential value in medical condition assessments. We propose a multi-modal conditional variational autoencoder (MC-VAE) capable of reconstructing features from missing modalities seen during training. We demonstrate that body positions can be effectively recognized from the available modality, achieving on par results with baseline models.
  arXiv  Detail & Related papers  (2021-11-30T04:56:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.