Digital Twin-Based 3D Map Management for Edge-Assisted Mobile Augmented Reality

• URL: http://arxiv.org/abs/2305.16571v1
• Date: Fri, 26 May 2023 01:38:45 GMT
• Title: Digital Twin-Based 3D Map Management for Edge-Assisted Mobile Augmented Reality
• Authors: Conghao Zhou, Jie Gao, Mushu Li, Nan Cheng, Xuemin Shen, Weihua Zhuang
• Abstract summary: We propose a digital twin (DT)-based approach to 3D map management for edge-assisted mobile augmented reality (MAR) First, a DT is created for the MAR device, which emulates 3D map management based on predicting subsequent camera frames. Second, a model-based reinforcement learning (MBRL) algorithm is developed, utilizing the data collected from both the actual and the emulated data to manage the 3D map.
• Score: 43.92003852614186
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this paper, we design a 3D map management scheme for edge-assisted mobile augmented reality (MAR) to support the pose estimation of individual MAR device, which uploads camera frames to an edge server. Our objective is to minimize the pose estimation uncertainty of the MAR device by periodically selecting a proper set of camera frames for uploading to update the 3D map. To address the challenges of the dynamic uplink data rate and the time-varying pose of the MAR device, we propose a digital twin (DT)-based approach to 3D map management. First, a DT is created for the MAR device, which emulates 3D map management based on predicting subsequent camera frames. Second, a model-based reinforcement learning (MBRL) algorithm is developed, utilizing the data collected from both the actual and the emulated data to manage the 3D map. With extensive emulated data provided by the DT, the MBRL algorithm can quickly provide an adaptive map management policy in a highly dynamic environment. Simulation results demonstrate that the proposed DT-based 3D map management outperforms benchmark schemes by achieving lower pose estimation uncertainty and higher data efficiency in dynamic environments.

Related papers

• POMATO: Marrying Pointmap Matching with Temporal Motion for Dynamic 3D Reconstruction [53.19968902152528]
  We present POMATO, a unified framework for dynamic 3D reconstruction by marrying pointmap matching with temporal motion. Specifically, our method learns an explicit matching relationship by mapping RGB pixels from both dynamic and static regions across different views to 3D pointmaps. We show the effectiveness of the proposed pointmap matching and temporal fusion paradigm by demonstrating the remarkable performance across multiple downstream tasks.
  arXiv  Detail & Related papers  (2025-04-08T05:33:13Z)
• Mapping and Localization Using LiDAR Fiducial Markers [0.8702432681310401]
  dissertation proposes a novel framework for mapping and localization using LiDAR fiducial markers. An Intensity Image-based LiDAR Fiducial Marker (IFM) system is introduced, using thin, letter-sized markers compatible with visual fiducial markers. New LFM-based mapping and localization method registers unordered, low-overlap point clouds.
  arXiv  Detail & Related papers  (2025-02-05T17:33:59Z)
• BIMCaP: BIM-based AI-supported LiDAR-Camera Pose Refinement [0.3749861135832073]
  This paper introduces BIMCaP, a novel method to integrate mobile 3D sparse LiDAR data and camera measurements with building information models. Experiments using real-world open-access data show that BIMCaP achieves superior accuracy, reducing translational error by over 4 cm.
  arXiv  Detail & Related papers  (2024-12-04T16:26:17Z)
• MM3DGS SLAM: Multi-modal 3D Gaussian Splatting for SLAM Using Vision, Depth, and Inertial Measurements [59.70107451308687]
  We show for the first time that using 3D Gaussians for map representation with unposed camera images and inertial measurements can enable accurate SLAM. Our method, MM3DGS, addresses the limitations of prior rendering by enabling faster scale awareness, and improved trajectory tracking. We also release a multi-modal dataset, UT-MM, collected from a mobile robot equipped with a camera and an inertial measurement unit.
  arXiv  Detail & Related papers  (2024-04-01T04:57:41Z)
• 3DGS-ReLoc: 3D Gaussian Splatting for Map Representation and Visual ReLocalization [13.868258945395326]
  This paper presents a novel system designed for 3D mapping and visual relocalization using 3D Gaussian Splatting. Our proposed method uses LiDAR and camera data to create accurate and visually plausible representations of the environment.
  arXiv  Detail & Related papers  (2024-03-17T23:06:12Z)
• Volumetric Semantically Consistent 3D Panoptic Mapping [77.13446499924977]
  We introduce an online 2D-to-3D semantic instance mapping algorithm aimed at generating semantic 3D maps suitable for autonomous agents in unstructured environments. It introduces novel ways of integrating semantic prediction confidence during mapping, producing semantic and instance-consistent 3D regions. The proposed method achieves accuracy superior to the state of the art on public large-scale datasets, improving on a number of widely used metrics.
  arXiv  Detail & Related papers  (2023-09-26T08:03:10Z)
• Poses as Queries: Image-to-LiDAR Map Localization with Transformers [5.704968411509063]
  High-precision vehicle localization with commercial setups is a crucial technique for high-level autonomous driving tasks. Estimate pose by finding correspondences between such cross-modal sensor data is challenging. We propose a novel Transformer-based neural network to register 2D images into 3D LiDAR map in an end-to-end manner.
  arXiv  Detail & Related papers  (2023-05-07T14:57:58Z)
• 3D Data Augmentation for Driving Scenes on Camera [50.41413053812315]
  We propose a 3D data augmentation approach termed Drive-3DAug, aiming at augmenting the driving scenes on camera in the 3D space. We first utilize Neural Radiance Field (NeRF) to reconstruct the 3D models of background and foreground objects. Then, augmented driving scenes can be obtained by placing the 3D objects with adapted location and orientation at the pre-defined valid region of backgrounds.
  arXiv  Detail & Related papers  (2023-03-18T05:51:05Z)
• Dense Deep Unfolding Network with 3D-CNN Prior for Snapshot Compressive Imaging [6.289143409131908]
  Snapshot imaging (SCI) aims to record three-dimensional signals via a two-dimensional camera. We present a novel dense deep unfolding network (DUN) with 3D-CNN prior for SCI. In order to promote network adaption, we propose a dense feature map compressive (DFMA) module.
  arXiv  Detail & Related papers  (2021-09-14T09:42:42Z)
• Monocular Quasi-Dense 3D Object Tracking [99.51683944057191]
  A reliable and accurate 3D tracking framework is essential for predicting future locations of surrounding objects and planning the observer's actions in numerous applications such as autonomous driving. We propose a framework that can effectively associate moving objects over time and estimate their full 3D bounding box information from a sequence of 2D images captured on a moving platform.
  arXiv  Detail & Related papers  (2021-03-12T15:30:02Z)
• Walk2Map: Extracting Floor Plans from Indoor Walk Trajectories [23.314557741879664]
  We present Walk2Map, a data-driven approach to generate floor plans from trajectories of a person walking inside the rooms. Thanks to advances in data-driven inertial odometry, such minimalistic input data can be acquired from the IMU readings of consumer-level smartphones. We train our networks using scanned 3D indoor models and apply them in a cascaded fashion on an indoor walk trajectory.
  arXiv  Detail & Related papers  (2021-02-27T16:29:09Z)
• Lightweight Multi-View 3D Pose Estimation through Camera-Disentangled Representation [57.11299763566534]
  We present a solution to recover 3D pose from multi-view images captured with spatially calibrated cameras. We exploit 3D geometry to fuse input images into a unified latent representation of pose, which is disentangled from camera view-points. Our architecture then conditions the learned representation on camera projection operators to produce accurate per-view 2d detections.
  arXiv  Detail & Related papers  (2020-04-05T12:52:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.