Designing Fatigue-Aware VR Interfaces via Biomechanical Models

• URL: http://arxiv.org/abs/2603.26031v1
• Date: Fri, 27 Mar 2026 03:00:17 GMT
• Title: Designing Fatigue-Aware VR Interfaces via Biomechanical Models
• Abstract summary: Prolonged mid-air interaction in virtual reality (VR) causes arm fatigue and discomfort, negatively affecting user experience.<n>We propose a hierarchical reinforcement learning framework that leverages biomechanical user models to evaluate and optimize VR interfaces for mid-air interaction.
• Score: 3.007949058551534
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Prolonged mid-air interaction in virtual reality (VR) causes arm fatigue and discomfort, negatively affecting user experience. Incorporating ergonomic considerations into VR user interface (UI) design typically requires extensive human-in-the-loop evaluation. Although biomechanical models have been used to simulate human behavior in HCI tasks, their application as surrogate users for ergonomic VR UI design remains underexplored. We propose a hierarchical reinforcement learning framework that leverages biomechanical user models to evaluate and optimize VR interfaces for mid-air interaction. A motion agent is trained to perform button-press tasks in VR under sequential conditions, using realistic movement strategies and estimating muscle-level effort via a validated three-compartment control with recovery (3CC-r) fatigue model. The simulated fatigue output serves as feedback for a UI agent that optimizes UI element layout via reinforcement learning (RL) to minimize fatigue. We compare the RL-optimized layout against a manually-designed centered baseline and a Bayesian optimized baseline. Results show that fatigue trends from the biomechanical model align with human user data. Moreover, the RL-optimized layout using simulated fatigue feedback produced significantly lower perceived fatigue in a follow-up human study. We further demonstrate the framework's extensibility via a simulated case study on longer sequential tasks with non-uniform interaction frequencies. To our knowledge, this is the first work using simulated biomechanical muscle fatigue as a direct optimization signal for VR UI layout design. Our findings highlight the potential of biomechanical user models as effective surrogate tools for ergonomic VR interface design, enabling efficient early-stage iteration with less reliance on extensive human participation.

Related papers

• A Replay-Constrained Simulation Framework for Personalization of Powered Knee--Ankle Prosthesis Controllers [11.765906112645466]
  Personalization of impedance controllers for powered prosthetic legs is critical to accommodating individual gait but remains challenging.<n>MuJoCo-based simulator reproduces prosthetic knee-ankle dynamics while replaying recorded hip kinematics and feedback-based ground reaction forces.<n>Deep reinforcement learning policy personalizes phase-dependent stiffness, damping, and equilibrium angle at both joints simultaneously.
  arXiv  Detail & Related papers  (2026-07-24T18:59:48Z)
• GS-Playground: A High-Throughput Photorealistic Simulator for Vision-Informed Robot Learning [55.272458304179025]
  GS-Playground is a multi-modal simulation framework designed to accelerate end-to-end perceptual learning.<n>Our system achieves a breakthrough throughput of 104 FPS at 640x480 resolution, significantly lowering the barrier for large-scale visual RL.
  arXiv  Detail & Related papers  (2026-04-28T10:05:39Z)
• Mind the Sim2Real Gap in User Simulation for Agentic Tasks [101.69142591891234]
  We formalize the Sim2Real gap in user simulation and present the first study running the full $$-bench protocol with real humans.<n> Behaviorally, LLM simulators are excessively cooperative, stylistically uniform, and lack realistic frustration or ambiguity.<n>These findings highlight the importance of human validation when using LLM-based user simulators in the agent development cycle.
  arXiv  Detail & Related papers  (2026-03-11T19:12:31Z)
• MOSAIC: Bridging the Sim-to-Real Gap in Generalist Humanoid Motion Tracking and Teleoperation with Rapid Residual Adaptation [50.643488406438514]
  Generalist humanoid motion trackers have recently achieved strong simulation metrics by scaling data and training, yet often remain brittle on hardware during sustained teleoperation.<n>We present MOSAIC, an open-source, full-stack system for humanoid motion tracking and whole-body teleoperation across multiple interfaces.
  arXiv  Detail & Related papers  (2026-02-09T12:35:01Z)
• InternVLA-A1: Unifying Understanding, Generation and Action for Robotic Manipulation [77.07565723756119]
  We present InternVLA-A1, a vision-language-action model with dynamic predictive capabilities.<n>We pre-train these models on heterogeneous data sources over real-world robot data, synthetic simulation data, and human videos.<n>We evaluate InternVLA-A1 on 12 real-world robotic tasks and a simulation benchmark.
  arXiv  Detail & Related papers  (2026-01-05T18:54:29Z)
• ResMimic: From General Motion Tracking to Humanoid Whole-body Loco-Manipulation via Residual Learning [59.64325421657381]
  Humanoid whole-body loco-manipulation promises transformative capabilities for daily service and warehouse tasks.<n>We introduce ResMimic, a two-stage residual learning framework for precise and expressive humanoid control from human motion data.<n>Results show substantial gains in task success, training efficiency, and robustness over strong baselines.
  arXiv  Detail & Related papers  (2025-10-06T17:47:02Z)
• Predicting User Grasp Intentions in Virtual Reality [0.0]
  We evaluate classification and regression approaches across 810 trials with varied object types, sizes, and manipulations.<n>Regression-based approaches demonstrate more robust performance, with timing errors within 0.25 seconds and distance errors around 5-20 cm.<n>Our results underscore the potential of machine learning models to enhance VR interactions.
  arXiv  Detail & Related papers  (2025-08-05T15:17:19Z)
• Tremor Reduction for Accessible Ray Based Interaction in VR Applications [0.0]
  Many traditional 2D interface interaction methods have been directly converted to work in a VR space with little alteration to the input mechanism. In this paper we propose the use of a low pass filter, to normalize user input noise, alleviating fine motor requirements during ray-based interaction.
  arXiv  Detail & Related papers  (2024-05-12T17:07:16Z)
• Toward Optimized VR/AR Ergonomics: Modeling and Predicting User Neck Muscle Contraction [21.654553113159665]
  We measure, model, and predict VR users' neck muscle contraction levels (MCL) while they move their heads to interact with the virtual environment. We develop a bio-physically inspired computational model to predict neck MCL under diverse head kinematic states. We hope this research will motivate new ergonomic-centered designs for VR/AR and interactive graphics applications.
  arXiv  Detail & Related papers  (2023-08-28T18:58:01Z)
• Force-Aware Interface via Electromyography for Natural VR/AR Interaction [69.1332992637271]
  We design a learning-based neural interface for natural and intuitive force inputs in VR/AR. We show that our interface can decode finger-wise forces in real-time with 3.3% mean error, and generalize to new users with little calibration. We envision our findings to push forward research towards more realistic physicality in future VR/AR.
  arXiv  Detail & Related papers  (2022-10-03T20:51:25Z)
• Building Trust in Autonomous Vehicles: Role of Virtual Reality Driving Simulators in HMI Design [8.39368916644651]
  We propose a methodology to validate the user experience in AVs based on continuous, objective information gathered from physiological signals. We applied this methodology to the design of a head-up display interface delivering visual cues about the vehicle's sensory and planning systems.
  arXiv  Detail & Related papers  (2020-07-27T08:42:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.