Translating Video Recordings of Mobile App Usages into Replayable Scenarios

• URL: http://arxiv.org/abs/2005.09057v1
• Date: Mon, 18 May 2020 20:11:36 GMT
• Title: Translating Video Recordings of Mobile App Usages into Replayable Scenarios
• Authors: Carlos Bernal-C\'ardenas, Nathan Cooper, Kevin Moran, Oscar Chaparro, Andrian Marcus and Denys Poshyvanyk
• Abstract summary: V2S is a lightweight, automated approach for translating video recordings of Android app usages into replayable scenarios. We performed an extensive evaluation of V2S involving 175 videos depicting 3,534 GUI-based actions collected from users exercising features and reproducing bugs from over 80 popular Android apps.
• Score: 24.992877070869177
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Screen recordings of mobile applications are easy to obtain and capture a wealth of information pertinent to software developers (e.g., bugs or feature requests), making them a popular mechanism for crowdsourced app feedback. Thus, these videos are becoming a common artifact that developers must manage. In light of unique mobile development constraints, including swift release cycles and rapidly evolving platforms, automated techniques for analyzing all types of rich software artifacts provide benefit to mobile developers. Unfortunately, automatically analyzing screen recordings presents serious challenges, due to their graphical nature, compared to other types of (textual) artifacts. To address these challenges, this paper introduces V2S, a lightweight, automated approach for translating video recordings of Android app usages into replayable scenarios. V2S is based primarily on computer vision techniques and adapts recent solutions for object detection and image classification to detect and classify user actions captured in a video, and convert these into a replayable test scenario. We performed an extensive evaluation of V2S involving 175 videos depicting 3,534 GUI-based actions collected from users exercising features and reproducing bugs from over 80 popular Android apps. Our results illustrate that V2S can accurately replay scenarios from screen recordings, and is capable of reproducing $\approx$ 89% of our collected videos with minimal overhead. A case study with three industrial partners illustrates the potential usefulness of V2S from the viewpoint of developers.

Related papers

• V^3: Viewing Volumetric Videos on Mobiles via Streamable 2D Dynamic Gaussians [53.614560799043545]
  V3 (Viewing Volumetric Videos) is a novel approach that enables high-quality mobile rendering through the streaming of dynamic Gaussians. Our key innovation is to view dynamic 3DGS as 2D videos, facilitating the use of hardware video codecs. As the first to stream dynamic Gaussians on mobile devices, our companion player offers users an unprecedented volumetric video experience.
  arXiv  Detail & Related papers  (2024-09-20T16:54:27Z)
• RACCooN: A Versatile Instructional Video Editing Framework with Auto-Generated Narratives [58.15403987979496]
  This paper proposes RACCooN, a versatile and user-friendly video-to-paragraph-to-video generative framework. Our video generative model incorporates auto-generated narratives or instructions to enhance the quality and accuracy of the generated content. The proposed framework demonstrates impressive versatile capabilities in video-to-paragraph generation, video content editing, and can be incorporated into other SoTA video generative models for further enhancement.
  arXiv  Detail & Related papers  (2024-05-28T17:46:36Z)
• Composed Video Retrieval via Enriched Context and Discriminative Embeddings [118.66322242183249]
  Composed video retrieval (CoVR) is a challenging problem in computer vision. We introduce a novel CoVR framework that leverages detailed language descriptions to explicitly encode query-specific contextual information. Our approach achieves gains as high as around 7% in terms of recall@K=1 score.
  arXiv  Detail & Related papers  (2024-03-25T17:59:03Z)
• Authentication and integrity of smartphone videos through multimedia container structure analysis [9.781421596580298]
  This work presents a novel technique to detect possible attacks against MP4, MOV, and 3GP format videos that affect their integrity and authenticity. The objectives of the proposal are to verify the integrity of videos, identify the source of acquisition and distinguish between original and manipulated videos.
  arXiv  Detail & Related papers  (2024-02-05T22:34:24Z)
• EasyVolcap: Accelerating Neural Volumetric Video Research [69.59671164891725]
  Volumetric video is a technology that digitally records dynamic events such as artistic performances, sporting events, and remote conversations. EasyVolcap is a Python & Pytorch library for unifying the process of multi-view data processing, 4D scene reconstruction, and efficient dynamic volumetric video rendering.
  arXiv  Detail & Related papers  (2023-12-11T17:59:46Z)
• Playable Video Generation [47.531594626822155]
  We aim at allowing a user to control the generated video by selecting a discrete action at every time step as when playing a video game. The difficulty of the task lies both in learning semantically consistent actions and in generating realistic videos conditioned on the user input. We propose a novel framework for PVG that is trained in a self-supervised manner on a large dataset of unlabelled videos.
  arXiv  Detail & Related papers  (2021-01-28T18:55:58Z)
• It Takes Two to Tango: Combining Visual and Textual Information for Detecting Duplicate Video-Based Bug Reports [19.289285682720177]
  This paper presents Tango, a duplicate detection technique that operates purely on video-based bug reports. We evaluate multiple configurations of Tango in a comprehensive empirical evaluation on 4,860 duplicate detection tasks. On average, Tango can reduce developer effort by over 60%, illustrating its practicality.
  arXiv  Detail & Related papers  (2021-01-22T16:36:19Z)
• TinyVIRAT: Low-resolution Video Action Recognition [70.37277191524755]
  In real-world surveillance environments, the actions in videos are captured at a wide range of resolutions. We introduce a benchmark dataset, TinyVIRAT, which contains natural low-resolution activities. We propose a novel method for recognizing tiny actions in videos which utilizes a progressive generative approach.
  arXiv  Detail & Related papers  (2020-07-14T21:09:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.