Side-channel Inference of User Activities in AR/VR Using GPU Profiling

• URL: http://arxiv.org/abs/2509.10703v1
• Date: Fri, 12 Sep 2025 21:44:56 GMT
• Title: Side-channel Inference of User Activities in AR/VR Using GPU Profiling
• Authors: Seonghun Son, Chandrika Mukherjee, Reham Mohamed Aburas, Berk Gulmezoglu, Z. Berkay Celik,
• Abstract summary: We present OVRWatcher, a novel side-channel primitive for AR/VR devices that infers user activities by monitoring low-resolution (1Hz) GPU usage via a background script.<n>OVRWatcher captures correlations between GPU metrics and 3D object interactions under varying speeds, distances, and rendering scenarios.<n>It achieves over 99% accuracy in app fingerprinting and over 98% accuracy in object-level inference.
• Score: 9.072390470827283
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Over the past decade, AR/VR devices have drastically changed how we interact with the digital world. Users often share sensitive information, such as their location, browsing history, and even financial data, within third-party apps installed on these devices, assuming a secure environment protected from malicious actors. Recent research has revealed that malicious apps can exploit such capabilities and monitor benign apps to track user activities, leveraging fine-grained profiling tools, such as performance counter APIs. However, app-to-app monitoring is not feasible on all AR/VR devices (e.g., Meta Quest), as a concurrent standalone app execution is disabled. In this paper, we present OVRWatcher, a novel side-channel primitive for AR/VR devices that infers user activities by monitoring low-resolution (1Hz) GPU usage via a background script, unlike prior work that relies on high-resolution profiling. OVRWatcher captures correlations between GPU metrics and 3D object interactions under varying speeds, distances, and rendering scenarios, without requiring concurrent app execution, access to application data, or additional SDK installations. We demonstrate the efficacy of OVRWatcher in fingerprinting both standalone AR/VR and WebXR applications. OVRWatcher also distinguishes virtual objects, such as products in immersive shopping apps selected by real users and the number of participants in virtual meetings, thereby revealing users' product preferences and potentially exposing confidential information from those meetings. OVRWatcher achieves over 99% accuracy in app fingerprinting and over 98% accuracy in object-level inference.

Related papers

• Video Object Recognition in Mobile Edge Networks: Local Tracking or Edge Detection? [57.000348519630286]
  Recent advances in mobile edge computing have made it possible to offload-intensive object detection to edge servers equipped with high-accuracy neural networks.<n>This hybrid approach offers a promising solution but introduces a new challenge: deciding when to perform edge detection versus local tracking.<n>We propose the LTED-Ada in single-device setting, a deep reinforcement learning-based algorithm that adaptively selects between local tracking and edge detection.
  arXiv  Detail & Related papers  (2025-11-25T04:54:51Z)
• Deep Learning-based Lightweight RGB Object Tracking for Augmented Reality Devices [2.3102477806624084]
  Augmented Reality (AR) applications require robust real-time tracking of objects in the user's environment to correctly overlay virtual content.<n>Recent advances in computer vision have produced highly accurate deep learning-based object trackers, but these models are typically too heavy in computation and memory for wearable AR devices.<n>We present a lightweight RGB object tracking algorithm designed specifically for resource-constrained AR platforms.
  arXiv  Detail & Related papers  (2025-10-04T02:39:55Z)
• FPI-Det: a face--phone Interaction Dataset for phone-use detection and understanding [20.181223336698675]
  Mobile devices have created new challenges for vision systems in safety monitoring, workplace productivity assessment, and attention management.<n>We introduce the FPI-Det, containing 22,879 images with synchronized annotations for faces and phones across workplace, education, transportation, and public scenarios.
  arXiv  Detail & Related papers  (2025-09-11T02:50:03Z)
• AUTOVR: Automated UI Exploration for Detecting Sensitive Data Flow Exposures in Virtual Reality Apps [31.735550965389482]
  We present AUTOVR, an automatic framework for dynamic UI and user event interaction in VR apps built on the Unity Engine.<n>Unlike conventional Android and GUI testers, AUTOVR analyzes the app's internal binary to reveal hidden events and resolves generative event dependencies.<n>Our empirical evaluation demonstrates AUTOVR's superior performance, triggering an order of magnitude of more sensitive data exposures and significantly enhancing the privacy of VR apps.
  arXiv  Detail & Related papers  (2025-08-17T00:22:58Z)
• Predicting 3D Motion from 2D Video for Behavior-Based VR Biometrics [7.609875877250929]
  We propose an approach that uses 2D body joints, acquired from the right side of the participants using an external 2D camera.<n>Our method uses the 2D data of body joints that are not tracked by the VR device to predict past and future 3D tracks of the right controller.
  arXiv  Detail & Related papers  (2025-02-05T02:19:23Z)
• GAZEploit: Remote Keystroke Inference Attack by Gaze Estimation from Avatar Views in VR/MR Devices [8.206832482042682]
  We unveil GAZEploit, a novel eye-tracking based attack specifically designed to exploit these eye-tracking information by leveraging the common use of virtual appearances in VR applications. Our research, involving 30 participants, achieved over 80% accuracy in keystroke inference. Our study also identified over 15 top-rated apps in the Apple Store as vulnerable to the GAZEploit attack, emphasizing the urgent need for bolstered security measures for this state-of-the-art VR/MR text entry method.
  arXiv  Detail & Related papers  (2024-09-12T15:11:35Z)
• Deep Motion Masking for Secure, Usable, and Scalable Real-Time Anonymization of Virtual Reality Motion Data [49.68609500290361]
  Recent studies have demonstrated that the motion tracking "telemetry" data used by nearly all VR applications is as uniquely identifiable as a fingerprint scan. We present in this paper a state-of-the-art VR identification model that can convincingly bypass known defensive countermeasures.
  arXiv  Detail & Related papers  (2023-11-09T01:34:22Z)
• SpikeMOT: Event-based Multi-Object Tracking with Sparse Motion Features [52.213656737672935]
  SpikeMOT is an event-based multi-object tracker. SpikeMOT uses spiking neural networks to extract sparsetemporal features from event streams associated with objects.
  arXiv  Detail & Related papers  (2023-09-29T05:13:43Z)
• EventTransAct: A video transformer-based framework for Event-camera based action recognition [52.537021302246664]
  Event cameras offer new opportunities compared to standard action recognition in RGB videos. In this study, we employ a computationally efficient model, namely the video transformer network (VTN), which initially acquires spatial embeddings per event-frame. In order to better adopt the VTN for the sparse and fine-grained nature of event data, we design Event-Contrastive Loss ($mathcalL_EC$) and event-specific augmentations.
  arXiv  Detail & Related papers  (2023-08-25T23:51:07Z)
• BehaVR: User Identification Based on VR Sensor Data [7.114684260471529]
  We introduce BehaVR, a framework for collecting and analyzing data from all sensor groups collected by multiple apps running on a VR device. We use BehaVR to collect data from real users that interact with 20 popular real-world apps. We build machine learning models for user identification within and across apps, with features extracted from available sensor data.
  arXiv  Detail & Related papers  (2023-08-14T17:43:42Z)
• Propagate And Calibrate: Real-time Passive Non-line-of-sight Tracking [84.38335117043907]
  We propose a purely passive method to track a person walking in an invisible room by only observing a relay wall. To excavate imperceptible changes in videos of the relay wall, we introduce difference frames as an essential carrier of temporal-local motion messages. To evaluate the proposed method, we build and publish the first dynamic passive NLOS tracking dataset, NLOS-Track.
  arXiv  Detail & Related papers  (2023-03-21T12:18:57Z)
• A Wireless-Vision Dataset for Privacy Preserving Human Activity Recognition [53.41825941088989]
  A new WiFi-based and video-based neural network (WiNN) is proposed to improve the robustness of activity recognition. Our results show that WiVi data set satisfies the primary demand and all three branches in the proposed pipeline keep more than $80%$ of activity recognition accuracy.
  arXiv  Detail & Related papers  (2022-05-24T10:49:11Z)
• Argus++: Robust Real-time Activity Detection for Unconstrained Video Streams with Overlapping Cube Proposals [85.76513755331318]
  Argus++ is a robust real-time activity detection system for analyzing unconstrained video streams. The overall system is optimized for real-time processing on standalone consumer-level hardware.
  arXiv  Detail & Related papers  (2022-01-14T03:35:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.