Large Model for Small Data: Foundation Model for Cross-Modal RF Human Activity Recognition

• URL: http://arxiv.org/abs/2410.19766v1
• Date: Sun, 13 Oct 2024 03:43:59 GMT
• Title: Large Model for Small Data: Foundation Model for Cross-Modal RF Human Activity Recognition
• Authors: Yuxuan Weng, Guoquan Wu, Tianyue Zheng, Yanbing Yang, Jun Luo,
• Abstract summary: We introduce FM-Fi, a cross-modal framework engineered to translate the knowledge of vision-based FMs for enhancing RF-based HAR systems. FM-Fi involves a novel cross-modal contrastive knowledge distillation mechanism, enabling an RF encoder to inherit the interpretative power of FMs. It also employs the intrinsic capabilities of FM and RF to remove extraneous features for better alignment between the two modalities.
• Score: 7.351361666395708
• License:
• Abstract: Radio-Frequency (RF)-based Human Activity Recognition (HAR) rises as a promising solution for applications unamenable to techniques requiring computer visions. However, the scarcity of labeled RF data due to their non-interpretable nature poses a significant obstacle. Thanks to the recent breakthrough of foundation models (FMs), extracting deep semantic insights from unlabeled visual data become viable, yet these vision-based FMs fall short when applied to small RF datasets. To bridge this gap, we introduce FM-Fi, an innovative cross-modal framework engineered to translate the knowledge of vision-based FMs for enhancing RF-based HAR systems. FM-Fi involves a novel cross-modal contrastive knowledge distillation mechanism, enabling an RF encoder to inherit the interpretative power of FMs for achieving zero-shot learning. It also employs the intrinsic capabilities of FM and RF to remove extraneous features for better alignment between the two modalities. The framework is further refined through metric-based few-shot learning techniques, aiming to boost the performance for predefined HAR tasks. Comprehensive evaluations evidently indicate that FM-Fi rivals the effectiveness of vision-based methodologies, and the evaluation results provide empirical validation of FM-Fi's generalizability across various environments.

Related papers

• Brain Network Diffusion-Driven fMRI Connectivity Augmentation for Enhanced Autism Spectrum Disorder Diagnosis [12.677178802864029]
  Due to the high cost of fMRI data acquisition and labeling, the amount of fMRI data is usually small. With the rise of generative models, especially diffusion models, the ability to generate realistic samples close to the real data distribution has been widely used for data augmentations.
  arXiv  Detail & Related papers  (2024-09-11T08:02:57Z)
• NeRF-DetS: Enhancing Multi-View 3D Object Detection with Sampling-adaptive Network of Continuous NeRF-based Representation [60.47114985993196]
  NeRF-Det unifies the tasks of novel view arithmetic and 3D perception. We introduce a novel 3D perception network structure, NeRF-DetS. NeRF-DetS outperforms competitive NeRF-Det on the ScanNetV2 dataset.
  arXiv  Detail & Related papers  (2024-04-22T06:59:03Z)
• RF-Diffusion: Radio Signal Generation via Time-Frequency Diffusion [15.175370227353406]
  We introduce a novel Time-Frequency Diffusion theory to enhance the original diffusion model, enabling it to tap into the information within the time, frequency, and complex-valued domains of RF signals. RF-Diffusion is a versatile solution to generate diverse, high-quality, and time-series RF data. We also showcase the versatility of RF-Diffusion in boosting Wi-Fi sensing systems and performing channel estimation in 5G networks.
  arXiv  Detail & Related papers  (2024-04-14T04:56:05Z)
• Foundation Model Sherpas: Guiding Foundation Models through Knowledge and Reasoning [23.763256908202496]
  Foundation models (FMs) have revolutionized the field of AI by showing remarkable performance in various tasks. FMs exhibit numerous limitations that prevent their broader adoption in many real-world systems. We propose a conceptual framework that encapsulates different modes by which agents could interact with FMs.
  arXiv  Detail & Related papers  (2024-02-02T18:00:35Z)
• A Survey on Efficient Federated Learning Methods for Foundation Model Training [62.473245910234304]
  Federated Learning (FL) has become an established technique to facilitate privacy-preserving collaborative training across a multitude of clients. In the wake of Foundation Models (FM), the reality is different for many deep learning applications. We discuss the benefits and drawbacks of parameter-efficient fine-tuning (PEFT) for FL applications.
  arXiv  Detail & Related papers  (2024-01-09T10:22:23Z)
• Robust and Communication-Efficient Federated Domain Adaptation via Random Features [9.97347047837426]
  federated domain adaptation (FDA) emerges as a powerful approach to address this challenge. RF-TCA is an enhancement to the standard Transfer Component Analysis approach that significantly accelerates computation without compromising theoretical and empirical performance. We present extensive experiments to showcase the superior performance and robustness (to network condition) of FedRF-TCA.
  arXiv  Detail & Related papers  (2023-11-08T13:46:58Z)
• Learn From Model Beyond Fine-Tuning: A Survey [78.80920533793595]
  Learn From Model (LFM) focuses on the research, modification, and design of foundation models (FM) based on the model interface. The study of LFM techniques can be broadly categorized into five major areas: model tuning, model distillation, model reuse, meta learning and model editing. This paper gives a comprehensive review of the current methods based on FM from the perspective of LFM.
  arXiv  Detail & Related papers  (2023-10-12T10:20:36Z)
• The Role of Federated Learning in a Wireless World with Foundation Models [59.8129893837421]
  Foundation models (FMs) are general-purpose artificial intelligence (AI) models that have recently enabled multiple brand-new generative AI applications. Currently, the exploration of the interplay between FMs and federated learning (FL) is still in its nascent stage. This article explores the extent to which FMs are suitable for FL over wireless networks, including a broad overview of research challenges and opportunities.
  arXiv  Detail & Related papers  (2023-10-06T04:13:10Z)
• Boosting Factorization Machines via Saliency-Guided Mixup [125.15872106335692]
  We present MixFM, inspired by Mixup, to generate auxiliary training data to boost Factorization machines (FMs) We also put forward a novel Factorization Machine powered by Saliency-guided Mixup (denoted as SMFM)
  arXiv  Detail & Related papers  (2022-06-17T09:49:00Z)
• Model-based Deep Learning Receiver Design for Rate-Splitting Multiple Access [65.21117658030235]
  This work proposes a novel design for a practical RSMA receiver based on model-based deep learning (MBDL) methods. The MBDL receiver is evaluated in terms of uncoded Symbol Error Rate (SER), throughput performance through Link-Level Simulations (LLS) and average training overhead. Results reveal that the MBDL outperforms by a significant margin the SIC receiver with imperfect CSIR.
  arXiv  Detail & Related papers  (2022-05-02T12:23:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.