Self-Supervised Radio-Visual Representation Learning for 6G Sensing
- URL: http://arxiv.org/abs/2111.02887v1
- Date: Mon, 1 Nov 2021 12:23:47 GMT
- Title: Self-Supervised Radio-Visual Representation Learning for 6G Sensing
- Authors: Mohammed Alloulah, Akash Deep Singh, Maximilian Arnold
- Abstract summary: In future 6G cellular networks, a joint communication and sensing protocol will allow the network to perceive the environment.
We propose to combine radio and vision to automatically learn a radio-only sensing model with minimal human intervention.
- Score: 1.9766522384767227
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: In future 6G cellular networks, a joint communication and sensing protocol
will allow the network to perceive the environment, opening the door for many
new applications atop a unified communication-perception infrastructure.
However, interpreting the sparse radio representation of sensing scenes is
challenging, which hinders the potential of these emergent systems. We propose
to combine radio and vision to automatically learn a radio-only sensing model
with minimal human intervention. We want to build a radio sensing model that
can feed on millions of uncurated data points. To this end, we leverage recent
advances in self-supervised learning and formulate a new label-free
radio-visual co-learning scheme, whereby vision trains radio via cross-modal
mutual information. We implement and evaluate our scheme according to the
common linear classification benchmark, and report qualitative and quantitative
performance metrics. In our evaluation, the representation learnt by
radio-visual self-supervision works well for a downstream sensing demonstrator,
and outperforms its fully-supervised counterpart when less labelled data is
used. This indicates that self-supervised learning could be an important
enabler for future scalable radio sensing systems.
Related papers
- Semantic Communication for Cooperative Perception using HARQ [51.148203799109304]
We leverage an importance map to distill critical semantic information, introducing a cooperative perception semantic communication framework.
To counter the challenges posed by time-varying multipath fading, our approach incorporates the use of frequency-division multiplexing (OFDM) along with channel estimation and equalization strategies.
We introduce a novel semantic error detection method that is integrated with our semantic communication framework in the spirit of hybrid automatic repeated request (HARQ)
arXiv Detail & Related papers (2024-08-29T08:53:26Z) - Fast and Accurate Cooperative Radio Map Estimation Enabled by GAN [63.90647197249949]
In the 6G era, real-time radio resource monitoring and management are urged to support diverse wireless-empowered applications.
In this paper, we present a cooperative radio map estimation approach enabled by the generative adversarial network (GAN)
arXiv Detail & Related papers (2024-02-05T05:01:28Z) - Physical-Layer Semantic-Aware Network for Zero-Shot Wireless Sensing [74.12670841657038]
Device-free wireless sensing has recently attracted significant interest due to its potential to support a wide range of immersive human-machine interactive applications.
Data heterogeneity in wireless signals and data privacy regulation of distributed sensing have been considered as the major challenges that hinder the wide applications of wireless sensing in large area networking systems.
We propose a novel zero-shot wireless sensing solution that allows models constructed in one or a limited number of locations to be directly transferred to other locations without any labeled data.
arXiv Detail & Related papers (2023-12-08T13:50:30Z) - Look, Radiate, and Learn: Self-Supervised Localisation via Radio-Visual
Correspondence [1.6219158909792257]
Next generation cellular networks will implement radio sensing functions alongside customary communications.
We present MaxRay: a synthetic radio-visual dataset and benchmark that facilitate precise target localisation in radio.
We use such self-supervised coordinates to train a radio localiser network.
arXiv Detail & Related papers (2022-06-13T19:08:36Z) - Multi-task Learning Approach for Modulation and Wireless Signal
Classification for 5G and Beyond: Edge Deployment via Model Compression [1.218340575383456]
Future communication networks must address the scarce spectrum to accommodate growth of heterogeneous wireless devices.
We exploit the potential of deep neural networks based multi-task learning framework to simultaneously learn modulation and signal classification tasks.
We provide a comprehensive heterogeneous wireless signals dataset for public use.
arXiv Detail & Related papers (2022-02-26T14:51:02Z) - Self-supervised Audiovisual Representation Learning for Remote Sensing Data [96.23611272637943]
We propose a self-supervised approach for pre-training deep neural networks in remote sensing.
By exploiting the correspondence between geo-tagged audio recordings and remote sensing, this is done in a completely label-free manner.
We show that our approach outperforms existing pre-training strategies for remote sensing imagery.
arXiv Detail & Related papers (2021-08-02T07:50:50Z) - Distributed Learning in Wireless Networks: Recent Progress and Future
Challenges [170.35951727508225]
Next-generation wireless networks will enable many machine learning (ML) tools and applications to analyze various types of data collected by edge devices.
Distributed learning and inference techniques have been proposed as a means to enable edge devices to collaboratively train ML models without raw data exchanges.
This paper provides a comprehensive study of how distributed learning can be efficiently and effectively deployed over wireless edge networks.
arXiv Detail & Related papers (2021-04-05T20:57:56Z) - Vision-Aided Radio: User Identity Match in Radio and Video Domains Using
Machine Learning [3.0204520109309847]
5G is designed to be an essential enabler and a leading infrastructure provider in the communication technology industry.
The use of deep learning and computer vision tools has the means to increase the environmental awareness of the network.
We propose a framework to match the information from both visual and radio domains.
arXiv Detail & Related papers (2020-10-14T16:32:22Z) - Federated Self-Supervised Learning of Multi-Sensor Representations for
Embedded Intelligence [8.110949636804772]
Smartphones, wearables, and Internet of Things (IoT) devices produce a wealth of data that cannot be accumulated in a centralized repository for learning supervised models.
We propose a self-supervised approach termed textitscalogram-signal correspondence learning based on wavelet transform to learn useful representations from unlabeled sensor inputs.
We extensively assess the quality of learned features with our multi-view strategy on diverse public datasets, achieving strong performance in all domains.
arXiv Detail & Related papers (2020-07-25T21:59:17Z) - X-ModalNet: A Semi-Supervised Deep Cross-Modal Network for
Classification of Remote Sensing Data [69.37597254841052]
We propose a novel cross-modal deep-learning framework called X-ModalNet.
X-ModalNet generalizes well, owing to propagating labels on an updatable graph constructed by high-level features on the top of the network.
We evaluate X-ModalNet on two multi-modal remote sensing datasets (HSI-MSI and HSI-SAR) and achieve a significant improvement in comparison with several state-of-the-art methods.
arXiv Detail & Related papers (2020-06-24T15:29:41Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.