Mobile Edge Adversarial Detection for Digital Twinning to the Metaverse with Deep Reinforcement Learning

• URL: http://arxiv.org/abs/2303.10288v1
• Date: Sat, 18 Mar 2023 00:03:50 GMT
• Title: Mobile Edge Adversarial Detection for Digital Twinning to the Metaverse with Deep Reinforcement Learning
• Authors: Terence Jie Chua, Wenhan Yu, Jun Zhao
• Abstract summary: Real-time Digital Twinning of physical world scenes onto the Metaverse is necessary for a myriad of applications such as augmented-reality (AR) assisted driving. In AR assisted driving, physical environment scenes are first captured by Internet of Vehicles (IoVs) and are uploaded to the Metaverse. A central Metaverse Map Service Provider (MMSP) will aggregate information from all IoVs to develop a central Metaverse Map. Information from the Metaverse Map can then be downloaded into individual IoVs on demand and be delivered as AR scenes to the driver.
• Score: 8.513938423514636
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Real-time Digital Twinning of physical world scenes onto the Metaverse is necessary for a myriad of applications such as augmented-reality (AR) assisted driving. In AR assisted driving, physical environment scenes are first captured by Internet of Vehicles (IoVs) and are uploaded to the Metaverse. A central Metaverse Map Service Provider (MMSP) will aggregate information from all IoVs to develop a central Metaverse Map. Information from the Metaverse Map can then be downloaded into individual IoVs on demand and be delivered as AR scenes to the driver. However, the growing interest in developing AR assisted driving applications which relies on digital twinning invites adversaries. These adversaries may place physical adversarial patches on physical world objects such as cars, signboards, or on roads, seeking to contort the virtual world digital twin. Hence, there is a need to detect these physical world adversarial patches. Nevertheless, as real-time, accurate detection of adversarial patches is compute-intensive, these physical world scenes have to be offloaded to the Metaverse Map Base Stations (MMBS) for computation. Hence in our work, we considered an environment with moving Internet of Vehicles (IoV), uploading real-time physical world scenes to the MMBSs. We formulated a realistic joint variable optimization problem where the MMSPs' objective is to maximize adversarial patch detection mean average precision (mAP), while minimizing the computed AR scene up-link transmission latency and IoVs' up-link transmission idle count, through optimizing the IoV-MMBS allocation and IoV up-link scene resolution selection. We proposed a Heterogeneous Action Proximal Policy Optimization (HAPPO) (discrete-continuous) algorithm to tackle the proposed problem. Extensive experiments shows HAPPO outperforms baseline models when compared against key metrics.

Related papers

• Blockchain-based Pseudonym Management for Vehicle Twin Migrations in Vehicular Edge Metaverse [73.79237826420925]
  Vehicle Twins (VTs) provide valuable metaverse services to improve driving safety and on-board satisfaction for VMUs throughout journeys. To maintain uninterrupted metaverse experiences, VTs must be migrated among edge servers following the movements of vehicles. This can raise concerns about privacy breaches during the dynamic communications among vehicular edge metaverses. Existing pseudonym management methods fall short in meeting the extensive pseudonym demands in vehicular edge metaverses.
  arXiv  Detail & Related papers  (2024-03-22T15:31:37Z)
• Distributed Radiance Fields for Edge Video Compression and Metaverse Integration in Autonomous Driving [13.536641570721798]
  metaverse is a virtual space that combines physical and digital elements, creating immersive and connected digital worlds. Digital twins (DTs) offer virtual prototyping, prediction, and more. DTs can be created with 3D scene reconstruction methods that capture the real world's geometry, appearance, and dynamics.
  arXiv  Detail & Related papers  (2024-02-22T15:39:58Z)
• Generative AI-empowered Effective Physical-Virtual Synchronization in the Vehicular Metaverse [129.8037449161817]
  We propose a generative AI-empowered physical-virtual synchronization framework for the vehicular Metaverse. In virtual-to-physical synchronization, MARs customize diverse and personal AR recommendations via generative AI models based on user preferences.
  arXiv  Detail & Related papers  (2023-01-18T16:25:42Z)
• Mobile Edge Computing for the Metaverse [15.738852406775289]
  The Metaverse has emerged as the next generation of the Internet. It aims to provide an immersive, persistent virtual space where people can live, learn, work and interact with each other. Existing technology is inadequate to guarantee high visual quality and ultra-low latency service for the Metaverse players. Mobile Edge Computing (MEC) is a paradigm where proximal edge servers are utilized to perform computation-intensive and latency-sensitive tasks like image processing and video analysis.
  arXiv  Detail & Related papers  (2022-12-19T03:37:32Z)
• LaMAR: Benchmarking Localization and Mapping for Augmented Reality [80.23361950062302]
  We introduce LaMAR, a new benchmark with a comprehensive capture and GT pipeline that co-registers realistic trajectories and sensor streams captured by heterogeneous AR devices. We publish a benchmark dataset of diverse and large-scale scenes recorded with head-mounted and hand-held AR devices.
  arXiv  Detail & Related papers  (2022-10-19T17:58:17Z)
• Resource Allocation for Mobile Metaverse with the Internet of Vehicles over 6G Wireless Communications: A Deep Reinforcement Learning Approach [8.513938423514636]
  The Metaverse relies on a core approach, digital twinning, which is a means to replicate physical world objects, people, actions and scenes onto the virtual world. With the development of Mobile Augmented Reality (MAR), users can interact via the Metaverse in a highly interactive manner, even under mobility. We design an environment with multiple cell stations, where there will be a handover of users' virtual world graphic download tasks between cell stations.
  arXiv  Detail & Related papers  (2022-09-27T14:28:04Z)
• Towards Scale Consistent Monocular Visual Odometry by Learning from the Virtual World [83.36195426897768]
  We propose VRVO, a novel framework for retrieving the absolute scale from virtual data. We first train a scale-aware disparity network using both monocular real images and stereo virtual data. The resulting scale-consistent disparities are then integrated with a direct VO system.
  arXiv  Detail & Related papers  (2022-03-11T01:51:54Z)
• Evaluating the Robustness of Semantic Segmentation for Autonomous Driving against Real-World Adversarial Patch Attacks [62.87459235819762]
  In a real-world scenario like autonomous driving, more attention should be devoted to real-world adversarial examples (RWAEs) This paper presents an in-depth evaluation of the robustness of popular SS models by testing the effects of both digital and real-world adversarial patches.
  arXiv  Detail & Related papers  (2021-08-13T11:49:09Z)
• Multi-Modal Fusion Transformer for End-to-End Autonomous Driving [59.60483620730437]
  We propose TransFuser, a novel Multi-Modal Fusion Transformer, to integrate image and LiDAR representations using attention. Our approach achieves state-of-the-art driving performance while reducing collisions by 76% compared to geometry-based fusion.
  arXiv  Detail & Related papers  (2021-04-19T11:48:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.