Look, Radiate, and Learn: Self-Supervised Localisation via Radio-Visual Correspondence

• URL: http://arxiv.org/abs/2206.06424v4
• Date: Wed, 29 Mar 2023 10:11:26 GMT
• Title: Look, Radiate, and Learn: Self-Supervised Localisation via Radio-Visual Correspondence
• Authors: Mohammed Alloulah, Maximilian Arnold
• Abstract summary: Next generation cellular networks will implement radio sensing functions alongside customary communications. We present MaxRay: a synthetic radio-visual dataset and benchmark that facilitate precise target localisation in radio. We use such self-supervised coordinates to train a radio localiser network.
• Score: 1.6219158909792257
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Next generation cellular networks will implement radio sensing functions alongside customary communications, thereby enabling unprecedented worldwide sensing coverage outdoors. Deep learning has revolutionised computer vision but has had limited application to radio perception tasks, in part due to lack of systematic datasets and benchmarks dedicated to the study of the performance and promise of radio sensing. To address this gap, we present MaxRay: a synthetic radio-visual dataset and benchmark that facilitate precise target localisation in radio. We further propose to learn to localise targets in radio without supervision by extracting self-coordinates from radio-visual correspondence. We use such self-supervised coordinates to train a radio localiser network. We characterise our performance against a number of state-of-the-art baselines. Our results indicate that accurate radio target localisation can be automatically learned from paired radio-visual data without labels, which is important for empirical data. This opens the door for vast data scalability and may prove key to realising the promise of robust radio sensing atop a unified communication-perception cellular infrastructure. Dataset will be hosted on IEEE DataPort.

Related papers

• Radio U-Net: a convolutional neural network to detect diffuse radio sources in galaxy clusters and beyond [0.0]
  Radio interferometric images of diffuse sources present a challenge for image segmentation tasks. We introduce Radio U-Net, a fully convolutional neural network based on the U-Net architecture. Radio U-Net is designed to detect faint and extended sources in radio surveys.
  arXiv  Detail & Related papers  (2024-08-20T14:03:21Z)
• RayProNet: A Neural Point Field Framework for Radio Propagation Modeling in 3D Environments [1.7074276434401858]
  We introduce a novel machine learning-empowered methodology for wireless channel modeling. Key ingredients include a point-cloud-based neural network and a Spherical Harmonics encoder with light probes.
  arXiv  Detail & Related papers  (2024-06-04T01:06:41Z)
• RadioGAT: A Joint Model-based and Data-driven Framework for Multi-band Radiomap Reconstruction via Graph Attention Networks [36.8227064106456]
  Multi-band radiomap reconstruction (MB-RMR) is a key component in wireless communications for tasks such as spectrum management and network planning. Traditional machine-learning-based MB-RMR methods, which rely heavily on simulated data, face significant deployment challenges. This study presents RadioGAT, a novel framework based on Graph Attention Network (GAT) tailored for MB-RMR within a single area.
  arXiv  Detail & Related papers  (2024-03-25T03:23:10Z)
• Fast and Accurate Cooperative Radio Map Estimation Enabled by GAN [63.90647197249949]
  In the 6G era, real-time radio resource monitoring and management are urged to support diverse wireless-empowered applications. In this paper, we present a cooperative radio map estimation approach enabled by the generative adversarial network (GAN)
  arXiv  Detail & Related papers  (2024-02-05T05:01:28Z)
• Radio Map Estimation -- An Open Dataset with Directive Transmitter Antennas and Initial Experiments [49.61405888107356]
  We release a dataset of simulated path loss radio maps together with realistic city maps from real-world locations and aerial images from open datasources. Initial experiments regarding model architectures, input feature design and estimation of radio maps from aerial images are presented.
  arXiv  Detail & Related papers  (2024-01-12T14:56:45Z)
• UAV-aided RF Mapping for Sensing and Connectivity in Wireless Networks [52.14281905671453]
  The use of unmanned aerial vehicles (UAV) as flying radio access network (RAN) nodes offers a promising complement to traditional fixed terrestrial deployments. Radio mapping is one of the challenges related to this task, referred here as radio mapping. The advantages induced by radio-mapping in terms of connectivity, sensing, and localization performance are illustrated.
  arXiv  Detail & Related papers  (2022-05-06T16:16:08Z)
• Radio-Assisted Human Detection [61.738482870059805]
  We propose a radio-assisted human detection framework by incorporating radio information into the state-of-the-art detection methods. We extract the radio localization and identifer information from the radio signals to assist the human detection. Experiments on the simulative Microsoft COCO dataset and Caltech pedestrian datasets show that the mean average precision (mAP) and the miss rate can be improved with the aid of radio information.
  arXiv  Detail & Related papers  (2021-12-16T09:53:41Z)
• Three-Way Deep Neural Network for Radio Frequency Map Generation and Source Localization [67.93423427193055]
  Monitoring wireless spectrum over spatial, temporal, and frequency domains will become a critical feature in beyond-5G and 6G communication technologies. In this paper, we present a Generative Adversarial Network (GAN) machine learning model to interpolate irregularly distributed measurements across the spatial domain.
  arXiv  Detail & Related papers  (2021-11-23T22:25:10Z)
• Self-Supervised Radio-Visual Representation Learning for 6G Sensing [1.9766522384767227]
  In future 6G cellular networks, a joint communication and sensing protocol will allow the network to perceive the environment. We propose to combine radio and vision to automatically learn a radio-only sensing model with minimal human intervention.
  arXiv  Detail & Related papers  (2021-11-01T12:23:47Z)
• OpenCSI: An Open-Source Dataset for Indoor Localization Using CSI-Based Fingerprinting [73.9222625243696]
  Fingerprint-based localization methods propose a solution to this problem, but rely on a radio map that is effort-intensive to acquire. We automate the radio map acquisition phase using a software-defined radio (SDR) and a wheeled robot. We describe first localization experiments on this radio map using a convolutional neural network to regress for location coordinates.
  arXiv  Detail & Related papers  (2021-04-16T08:31:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.