Short-Term Trajectory Prediction for Full-Immersive Multiuser Virtual Reality with Redirected Walking

• URL: http://arxiv.org/abs/2207.07520v1
• Date: Fri, 15 Jul 2022 15:09:07 GMT
• Title: Short-Term Trajectory Prediction for Full-Immersive Multiuser Virtual Reality with Redirected Walking
• Authors: Filip Lemic, Jakob Struye, Jeroen Famaey
• Abstract summary: Full-immersive multiuser Virtual Reality (VR) envisions supporting unconstrained mobility of the users in the virtual worlds. We show that Gated Recurrent Unit (GRU) networks, another candidate from the RNN family, generally outperform the traditionally utilized LSTMs. Second, we show that context from a virtual world can enhance the accuracy of the prediction if used as an additional input feature.
• Score: 6.622115542749609
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Full-immersive multiuser Virtual Reality (VR) envisions supporting unconstrained mobility of the users in the virtual worlds, while at the same time constraining their physical movements inside VR setups through redirected walking. For enabling delivery of high data rate video content in real-time, the supporting wireless networks will leverage highly directional communication links that will "track" the users for maintaining the Line-of-Sight (LoS) connectivity. Recurrent Neural Networks (RNNs) and in particular Long Short-Term Memory (LSTM) networks have historically presented themselves as a suitable candidate for near-term movement trajectory prediction for natural human mobility, and have also recently been shown as applicable in predicting VR users' mobility under the constraints of redirected walking. In this work, we extend these initial findings by showing that Gated Recurrent Unit (GRU) networks, another candidate from the RNN family, generally outperform the traditionally utilized LSTMs. Second, we show that context from a virtual world can enhance the accuracy of the prediction if used as an additional input feature in comparison to the more traditional utilization of solely the historical physical movements of the VR users. Finally, we show that the prediction system trained on a static number of coexisting VR users be scaled to a multi-user system without significant accuracy degradation.

Related papers

• Human-Aware Vision-and-Language Navigation: Bridging Simulation to Reality with Dynamic Human Interactions [69.9980759344628]
  Vision-and-Language Navigation (VLN) aims to develop embodied agents that navigate based on human instructions. We introduce Human-Aware Vision-and-Language Navigation (HA-VLN), extending traditional VLN by incorporating dynamic human activities. We present the Expert-Supervised Cross-Modal (VLN-CM) and Non-Expert-Supervised Decision Transformer (VLN-DT) agents, utilizing cross-modal fusion and diverse training strategies.
  arXiv  Detail & Related papers  (2024-06-27T15:01:42Z)
• Predictive Context-Awareness for Full-Immersive Multiuser Virtual Reality with Redirected Walking [5.393569497095572]
  Future VR systems will require supporting wireless networking infrastructures operating in millimeter Wave (mmWave) frequencies. We propose the use of predictive context-awareness to optimize transmitter and receiver-side beamforming and beamsteering. We show that Long Short-Term Memory (LSTM) networks feature promising accuracy in predicting lateral movements.
  arXiv  Detail & Related papers  (2023-03-31T09:09:17Z)
• User-centric Heterogeneous-action Deep Reinforcement Learning for Virtual Reality in the Metaverse over Wireless Networks [8.513938423514636]
  In this paper, we consider a system consisting of a Metaverse server and multiple VR users. In our multi-user VR scenario for the Metaverse, users have different characteristics and demands for Frames Per Second (FPS) Our proposed user-centric DRL algorithm is called User-centric Critic with Heterogenous Actors (UCHA)
  arXiv  Detail & Related papers  (2023-02-03T00:12:12Z)
• Generative AI-empowered Effective Physical-Virtual Synchronization in the Vehicular Metaverse [129.8037449161817]
  We propose a generative AI-empowered physical-virtual synchronization framework for the vehicular Metaverse. In virtual-to-physical synchronization, MARs customize diverse and personal AR recommendations via generative AI models based on user preferences.
  arXiv  Detail & Related papers  (2023-01-18T16:25:42Z)
• WiserVR: Semantic Communication Enabled Wireless Virtual Reality Delivery [12.158124978097982]
  We propose a novel framework, namely WIreless SEmantic deliveRy for VR (WiserVR), for delivering consecutive 360deg video frames to VR users. Deep learning-based multiple modules are well-devised for the transceiver in WiserVR to realize high-performance feature extraction and semantic recovery.
  arXiv  Detail & Related papers  (2022-11-02T16:22:41Z)
• Force-Aware Interface via Electromyography for Natural VR/AR Interaction [69.1332992637271]
  We design a learning-based neural interface for natural and intuitive force inputs in VR/AR. We show that our interface can decode finger-wise forces in real-time with 3.3% mean error, and generalize to new users with little calibration. We envision our findings to push forward research towards more realistic physicality in future VR/AR.
  arXiv  Detail & Related papers  (2022-10-03T20:51:25Z)
• Motion Transformer with Global Intention Localization and Local Movement Refinement [103.75625476231401]
  Motion TRansformer (MTR) models motion prediction as the joint optimization of global intention localization and local movement refinement. MTR achieves state-of-the-art performance on both the marginal and joint motion prediction challenges.
  arXiv  Detail & Related papers  (2022-09-27T16:23:14Z)
• Wireless Edge-Empowered Metaverse: A Learning-Based Incentive Mechanism for Virtual Reality [102.4151387131726]
  We propose a learning-based Incentive Mechanism framework for VR services in the Metaverse. First, we propose the quality of perception as the metric for VR users in the virtual world. Second, for quick trading of VR services between VR users (i.e., buyers) and VR SPs (i.e., sellers), we design a double Dutch auction mechanism. Third, for auction communication reduction, we design a deep reinforcement learning-based auctioneer to accelerate this auction process.
  arXiv  Detail & Related papers  (2021-11-07T13:02:52Z)
• PlayVirtual: Augmenting Cycle-Consistent Virtual Trajectories for Reinforcement Learning [84.30765628008207]
  We propose a novel method, dubbed PlayVirtual, which augments cycle-consistent virtual trajectories to enhance the data efficiency for RL feature representation learning. Our method outperforms the current state-of-the-art methods by a large margin on both benchmarks.
  arXiv  Detail & Related papers  (2021-06-08T07:37:37Z)
• Hybrid Policy Learning for Energy-Latency Tradeoff in MEC-Assisted VR Video Service [35.31115954442725]
  We consider delivering the wireless multi-tile VR video service over a mobile edge computing network. We first cast the time-varying view popularity as a model-free Markov chain. A hybrid policy is then implemented to coordinate the dynamic caching replacement and the deterministic offloading.
  arXiv  Detail & Related papers  (2021-04-02T13:17:11Z)
• Learning-based Prediction and Uplink Retransmission for Wireless Virtual Reality (VR) Network [29.640073851481066]
  In this paper, we use offline and online learning algorithms to predict viewpoint of the VR user using real VR dataset. Our proposed online learning algorithm for uplink wireless VR network with the proactive retransmission scheme only exhibits about 5% prediction error.
  arXiv  Detail & Related papers  (2020-12-16T18:31:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.