User-centric Heterogeneous-action Deep Reinforcement Learning for Virtual Reality in the Metaverse over Wireless Networks

• URL: http://arxiv.org/abs/2302.01471v2
• Date: Mon, 22 May 2023 17:42:11 GMT
• Title: User-centric Heterogeneous-action Deep Reinforcement Learning for Virtual Reality in the Metaverse over Wireless Networks
• Authors: Wenhan Yu, Terence Jie Chua, Jun Zhao
• Abstract summary: In this paper, we consider a system consisting of a Metaverse server and multiple VR users. In our multi-user VR scenario for the Metaverse, users have different characteristics and demands for Frames Per Second (FPS) Our proposed user-centric DRL algorithm is called User-centric Critic with Heterogenous Actors (UCHA)
• Score: 8.513938423514636
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The Metaverse is emerging as maturing technologies are empowering the different facets. Virtual Reality (VR) technologies serve as the backbone of the virtual universe within the Metaverse to offer a highly immersive user experience. As mobility is emphasized in the Metaverse context, VR devices reduce their weights at the sacrifice of local computation abilities. In this paper, for a system consisting of a Metaverse server and multiple VR users, we consider two cases of (i) the server generating frames and transmitting them to users, and (ii) users generating frames locally and thus consuming device energy. Moreover, in our multi-user VR scenario for the Metaverse, users have different characteristics and demands for Frames Per Second (FPS). Then the channel access arrangement (including the decisions on frame generation location), and transmission powers for the downlink communications from the server to the users are jointly optimized to improve the utilities of users. This joint optimization is addressed by deep reinforcement learning (DRL) with heterogeneous actions. Our proposed user-centric DRL algorithm is called User-centric Critic with Heterogenous Actors (UCHA). Extensive experiments demonstrate that our UCHA algorithm leads to remarkable results under various requirements and constraints.

Related papers

• VR-GS: A Physical Dynamics-Aware Interactive Gaussian Splatting System in Virtual Reality [39.53150683721031]
  Our proposed VR-GS system represents a leap forward in human-centered 3D content interaction. The components of our Virtual Reality system are designed for high efficiency and effectiveness.
  arXiv  Detail & Related papers  (2024-01-30T01:28:36Z)
• Mobile Edge Computing and AI Enabled Web3 Metaverse over 6G Wireless Communications: A Deep Reinforcement Learning Approach [10.47302625959368]
  An interactive and immersive socialization experience is one of the promises of the Metaverse. The computation required for a smooth, seamless and immersive socialization experience in the Metaverse is overbearing. This paper introduces a novel Quality-of-Service (QoS) model for the accumulated experience in multi-user socialization on a multichannel wireless network.
  arXiv  Detail & Related papers  (2023-12-11T10:49:47Z)
• Vision-based Semantic Communications for Metaverse Services: A Contest Theoretic Approach [66.10465001046762]
  In Metaverse, avatars must be updated and rendered to reflect users' behaviour. We propose a semantic communication framework to model the interactions between users and MSPs. We use the semantic communication technique to reduce the amount of data to be transmitted.
  arXiv  Detail & Related papers  (2023-08-15T07:56:33Z)
• Predictive Context-Awareness for Full-Immersive Multiuser Virtual Reality with Redirected Walking [5.393569497095572]
  Future VR systems will require supporting wireless networking infrastructures operating in millimeter Wave (mmWave) frequencies. We propose the use of predictive context-awareness to optimize transmitter and receiver-side beamforming and beamsteering. We show that Long Short-Term Memory (LSTM) networks feature promising accuracy in predicting lateral movements.
  arXiv  Detail & Related papers  (2023-03-31T09:09:17Z)
• Virtual Reality in Metaverse over Wireless Networks with User-centered Deep Reinforcement Learning [8.513938423514636]
  We introduce a multi-user VR computation offloading over wireless communication scenario. In addition, we devised a novel user-centered deep reinforcement learning approach to find a near-optimal solution.
  arXiv  Detail & Related papers  (2023-03-08T03:10:41Z)
• Generative AI-empowered Effective Physical-Virtual Synchronization in the Vehicular Metaverse [129.8037449161817]
  We propose a generative AI-empowered physical-virtual synchronization framework for the vehicular Metaverse. In virtual-to-physical synchronization, MARs customize diverse and personal AR recommendations via generative AI models based on user preferences.
  arXiv  Detail & Related papers  (2023-01-18T16:25:42Z)
• Force-Aware Interface via Electromyography for Natural VR/AR Interaction [69.1332992637271]
  We design a learning-based neural interface for natural and intuitive force inputs in VR/AR. We show that our interface can decode finger-wise forces in real-time with 3.3% mean error, and generalize to new users with little calibration. We envision our findings to push forward research towards more realistic physicality in future VR/AR.
  arXiv  Detail & Related papers  (2022-10-03T20:51:25Z)
• Short-Term Trajectory Prediction for Full-Immersive Multiuser Virtual Reality with Redirected Walking [6.622115542749609]
  Full-immersive multiuser Virtual Reality (VR) envisions supporting unconstrained mobility of the users in the virtual worlds. We show that Gated Recurrent Unit (GRU) networks, another candidate from the RNN family, generally outperform the traditionally utilized LSTMs. Second, we show that context from a virtual world can enhance the accuracy of the prediction if used as an additional input feature.
  arXiv  Detail & Related papers  (2022-07-15T15:09:07Z)
• Wireless Edge-Empowered Metaverse: A Learning-Based Incentive Mechanism for Virtual Reality [102.4151387131726]
  We propose a learning-based Incentive Mechanism framework for VR services in the Metaverse. First, we propose the quality of perception as the metric for VR users in the virtual world. Second, for quick trading of VR services between VR users (i.e., buyers) and VR SPs (i.e., sellers), we design a double Dutch auction mechanism. Third, for auction communication reduction, we design a deep reinforcement learning-based auctioneer to accelerate this auction process.
  arXiv  Detail & Related papers  (2021-11-07T13:02:52Z)
• Feeling of Presence Maximization: mmWave-Enabled Virtual Reality Meets Deep Reinforcement Learning [76.46530937296066]
  This paper investigates the problem of providing ultra-reliable and energy-efficient virtual reality (VR) experiences for wireless mobile users. To ensure reliable ultra-high-definition (UHD) video frame delivery to mobile users, a coordinated multipoint (CoMP) transmission technique and millimeter wave (mmWave) communications are exploited.
  arXiv  Detail & Related papers  (2021-06-03T08:35:10Z)
• Meta-Reinforcement Learning for Reliable Communication in THz/VLC Wireless VR Networks [157.42035777757292]
  The problem of enhancing the quality of virtual reality (VR) services is studied for an indoor terahertz (THz)/visible light communication (VLC) wireless network. Small base stations (SBSs) transmit high-quality VR images to VR users over THz bands and light-emitting diodes (LEDs) provide accurate indoor positioning services. To control the energy consumption of the studied THz/VLC wireless VR network, VLC access points (VAPs) must be selectively turned on.
  arXiv  Detail & Related papers  (2021-01-29T15:57:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.