Towards Automated Accessibility Report Generation for Mobile Apps

• URL: http://arxiv.org/abs/2310.00091v2
• Date: Mon, 16 Oct 2023 18:29:18 GMT
• Title: Towards Automated Accessibility Report Generation for Mobile Apps
• Authors: Amanda Swearngin, Jason Wu, Xiaoyi Zhang, Esteban Gomez, Jen Coughenour, Rachel Stukenborg, Bhavya Garg, Greg Hughes, Adriana Hilliard, Jeffrey P. Bigham, Jeffrey Nichols
• Abstract summary: We propose a system to generate whole app accessibility reports. It combines varied data collection methods (e.g., app crawling, manual recording) with an existing accessibility scanner.
• Score: 14.908672785900832
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Many apps have basic accessibility issues, like missing labels or low contrast. Automated tools can help app developers catch basic issues, but can be laborious or require writing dedicated tests. We propose a system, motivated by a collaborative process with accessibility stakeholders at a large technology company, to generate whole app accessibility reports by combining varied data collection methods (e.g., app crawling, manual recording) with an existing accessibility scanner. Many such scanners are based on single-screen scanning, and a key problem in whole app accessibility reporting is to effectively de-duplicate and summarize issues collected across an app. To this end, we developed a screen grouping model with 96.9% accuracy (88.8% F1-score) and UI element matching heuristics with 97% accuracy (98.2% F1-score). We combine these technologies in a system to report and summarize unique issues across an app, and enable a unique pixel-based ignore feature to help engineers and testers better manage reported issues across their app's lifetime. We conducted a qualitative evaluation with 18 accessibility-focused engineers and testers which showed this system can enhance their existing accessibility testing toolkit and address key limitations in current accessibility scanning tools.

Related papers

• SPA-Bench: A Comprehensive Benchmark for SmartPhone Agent Evaluation [89.24729958546168]
  We present SPA-Bench, a comprehensive SmartPhone Agent Benchmark designed to evaluate (M)LLM-based agents. SPA-Bench offers three key contributions: A diverse set of tasks covering system and third-party apps in both English and Chinese, focusing on features commonly used in daily routines. A novel evaluation pipeline that automatically assesses agent performance across multiple dimensions, encompassing seven metrics related to task completion and resource consumption.
  arXiv  Detail & Related papers  (2024-10-19T17:28:48Z)
• Learning Visuotactile Skills with Two Multifingered Hands [80.99370364907278]
  We explore learning from human demonstrations using a bimanual system with multifingered hands and visuotactile data. Our results mark a promising step forward in bimanual multifingered manipulation from visuotactile data.
  arXiv  Detail & Related papers  (2024-04-25T17:59:41Z)
• MotorEase: Automated Detection of Motor Impairment Accessibility Issues in Mobile App UIs [8.057618278428494]
  MotorEase is capable of identifying accessibility issues in mobile app UIs that impact motor-impaired users. It adapts computer vision and text processing techniques to enable a semantic understanding of app UI screens. It is able to identify violations with an average accuracy of 90%, and a false positive rate of less than 9%.
  arXiv  Detail & Related papers  (2024-03-20T15:53:07Z)
• Fairness Concerns in App Reviews: A Study on AI-based Mobile Apps [9.948068408730654]
  This research aims to investigate fairness concerns raised in mobile app reviews. Our research focuses on AI-based mobile app reviews as the chance of unfair behaviors and outcomes in AI-based apps may be higher than in non-AI-based apps.
  arXiv  Detail & Related papers  (2024-01-16T03:43:33Z)
• Testing the Limits: Unusual Text Inputs Generation for Mobile App Crash Detection with Large Language Model [23.460051600514806]
  This paper proposes InputBlaster to automatically generate unusual text inputs for mobile app crash detection. It formulates the unusual inputs generation problem as a task of producing a set of test generators, each of which can yield a batch of unusual text inputs. It is evaluated on 36 text input widgets with cash bugs involving 31 popular Android apps, and results show that it achieves 78% bug detection rate, with 136% higher than the best baseline.
  arXiv  Detail & Related papers  (2023-10-24T09:10:51Z)
• Vision-Based Mobile App GUI Testing: A Survey [29.042723121518765]
  Vision-based mobile app GUI testing approaches emerged with the development of computer vision technologies. We provide a comprehensive investigation of the state-of-the-art techniques on 271 papers, among which 92 are vision-based studies.
  arXiv  Detail & Related papers  (2023-10-20T14:04:04Z)
• RH20T: A Comprehensive Robotic Dataset for Learning Diverse Skills in One-Shot [56.130215236125224]
  A key challenge in robotic manipulation in open domains is how to acquire diverse and generalizable skills for robots. Recent research in one-shot imitation learning has shown promise in transferring trained policies to new tasks based on demonstrations. This paper aims to unlock the potential for an agent to generalize to hundreds of real-world skills with multi-modal perception.
  arXiv  Detail & Related papers  (2023-07-02T15:33:31Z)
• Integrating Accessibility in a Mobile App Development Course [0.0]
  The course introduced three accessibility-related topics using various interventions: Accessibility Awareness (a guest lecture by a legal expert), Technical Knowledge (lectures on Android accessibility guidelines and testing practices), and Empathy (an activity that required students to blindfold themselves and interact with their phones using a screen-reader) All students could correctly identify at least one accessibility issue in the user interface of a real-world app given its screenshot, and 90% of them could provide a correct solution to fix it.
  arXiv  Detail & Related papers  (2022-10-12T12:44:33Z)
• Mobile Behavioral Biometrics for Passive Authentication [65.94403066225384]
  This work carries out a comparative analysis of unimodal and multimodal behavioral biometric traits. Experiments are performed over HuMIdb, one of the largest and most comprehensive freely available mobile user interaction databases. In our experiments, the most discriminative background sensor is the magnetometer, whereas among touch tasks the best results are achieved with keystroke.
  arXiv  Detail & Related papers  (2022-03-14T17:05:59Z)
• Emerging App Issue Identification via Online Joint Sentiment-Topic Tracing [66.57888248681303]
  We propose a novel emerging issue detection approach named MERIT. Based on the AOBST model, we infer the topics negatively reflected in user reviews for one app version. Experiments on popular apps from Google Play and Apple's App Store demonstrate the effectiveness of MERIT.
  arXiv  Detail & Related papers  (2020-08-23T06:34:05Z)
• Families In Wild Multimedia: A Multimodal Database for Recognizing Kinship [63.27052967981546]
  We introduce the first publicly available multi-task MM kinship dataset. To build FIW MM, we developed machinery to automatically collect, annotate, and prepare the data. Results highlight edge cases to inspire future research with different areas of improvement.
  arXiv  Detail & Related papers  (2020-07-28T22:36:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.