Sentiment Analysis of Users' Reviews on COVID-19 Contact Tracing Apps with a Benchmark Dataset

• URL: http://arxiv.org/abs/2103.01196v1
• Date: Mon, 1 Mar 2021 18:43:10 GMT
• Title: Sentiment Analysis of Users' Reviews on COVID-19 Contact Tracing Apps with a Benchmark Dataset
• Authors: Kashif Ahmad, Firoj Alam, Junaid Qadir, Basheer Qolomany, Imran Khan, Talhat Khan, Muhammad Suleman, Naina Said, Syed Zohaib Hassan, Asma Gul, Ala Al-Fuqaha
• Abstract summary: Contact tracing has been globally adopted in the fight to control the infection rate of COVID-19. Thanks to digital technologies, such as smartphones and wearable devices, contacts of COVID-19 patients can be easily traced and informed about their potential exposure to the virus. Several interesting mobile applications have been developed. However, there are ever-growing concerns over the working mechanism and performance of these applications. In this work, we propose a pipeline starting from manual annotation via a crowd-sourcing study and concluding on the development and training of AI models for automatic sentiment analysis of users' reviews.
• Score: 6.592595861973966
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Contact tracing has been globally adopted in the fight to control the infection rate of COVID-19. Thanks to digital technologies, such as smartphones and wearable devices, contacts of COVID-19 patients can be easily traced and informed about their potential exposure to the virus. To this aim, several interesting mobile applications have been developed. However, there are ever-growing concerns over the working mechanism and performance of these applications. The literature already provides some interesting exploratory studies on the community's response to the applications by analyzing information from different sources, such as news and users' reviews of the applications. However, to the best of our knowledge, there is no existing solution that automatically analyzes users' reviews and extracts the evoked sentiments. In this work, we propose a pipeline starting from manual annotation via a crowd-sourcing study and concluding on the development and training of AI models for automatic sentiment analysis of users' reviews. In total, we employ eight different methods achieving up to an average F1-Scores 94.8% indicating the feasibility of automatic sentiment analysis of users' reviews on the COVID-19 contact tracing applications. We also highlight the key advantages, drawbacks, and users' concerns over the applications. Moreover, we also collect and annotate a large-scale dataset composed of 34,534 reviews manually annotated from the contract tracing applications of 46 distinct countries. The presented analysis and the dataset are expected to provide a baseline/benchmark for future research in the domain.

Related papers

• Towards Extracting Ethical Concerns-related Software Requirements from App Reviews [0.0]
  This study analyzes app reviews of the Uber mobile application (a popular taxi/ride app) We propose a novel approach that leverages a knowledge graph (KG) model to extract software requirements from app reviews. Our framework consists of three main components: developing an ontology with relevant entities and relations, extracting key entities from app reviews, and creating connections between them.
  arXiv  Detail & Related papers  (2024-07-19T04:50:32Z)
• Is Reference Necessary in the Evaluation of NLG Systems? When and Where? [58.52957222172377]
  We show that reference-free metrics exhibit a higher correlation with human judgment and greater sensitivity to deficiencies in language quality. Our study can provide insight into the appropriate application of automatic metrics and the impact of metric choice on evaluation performance.
  arXiv  Detail & Related papers  (2024-03-21T10:31:11Z)
• Tool-Augmented Reward Modeling [58.381678612409]
  We propose a tool-augmented preference modeling approach, named Themis, to address limitations by empowering RMs with access to external environments. Our study delves into the integration of external tools into RMs, enabling them to interact with diverse external sources. In human evaluations, RLHF trained with Themis attains an average win rate of 32% when compared to baselines.
  arXiv  Detail & Related papers  (2023-10-02T09:47:40Z)
• PyRCA: A Library for Metric-based Root Cause Analysis [66.72542200701807]
  PyRCA is an open-source machine learning library of Root Cause Analysis (RCA) for Artificial Intelligence for IT Operations (AIOps) It provides a holistic framework to uncover the complicated metric causal dependencies and automatically locate root causes of incidents.
  arXiv  Detail & Related papers  (2023-06-20T09:55:10Z)
• A Unified Comparison of User Modeling Techniques for Predicting Data Interaction and Detecting Exploration Bias [17.518601254380275]
  We compare and rank eight user modeling algorithms based on their performance on a diverse set of four user study datasets. Based on our findings, we highlight open challenges and new directions for analyzing user interactions and visualization provenance.
  arXiv  Detail & Related papers  (2022-08-09T19:51:10Z)
• DAPPER: Label-Free Performance Estimation after Personalization for Heterogeneous Mobile Sensing [95.18236298557721]
  We present DAPPER (Domain AdaPtation Performance EstimatoR) that estimates the adaptation performance in a target domain with unlabeled target data. Our evaluation with four real-world sensing datasets compared against six baselines shows that DAPPER outperforms the state-of-the-art baseline by 39.8% in estimation accuracy.
  arXiv  Detail & Related papers  (2021-11-22T08:49:33Z)
• SIFN: A Sentiment-aware Interactive Fusion Network for Review-based Item Recommendation [48.1799451277808]
  We propose a Sentiment-aware Interactive Fusion Network (SIFN) for review-based item recommendation. We first encode user/item reviews via BERT and propose a light-weighted sentiment learner to extract semantic features of each review. Then, we propose a sentiment prediction task that guides the sentiment learner to extract sentiment-aware features via explicit sentiment labels.
  arXiv  Detail & Related papers  (2021-08-18T08:04:38Z)
• Project Achoo: A Practical Model and Application for COVID-19 Detection from Recordings of Breath, Voice, and Cough [55.45063681652457]
  We propose a machine learning method to quickly triage COVID-19 using recordings made on consumer devices. The approach combines signal processing methods with fine-tuned deep learning networks and provides methods for signal denoising, cough detection and classification. We have also developed and deployed a mobile application that uses symptoms checker together with voice, breath and cough signals to detect COVID-19 infection.
  arXiv  Detail & Related papers  (2021-07-12T08:07:56Z)
• The evolving ecosystem of COVID-19 contact tracing applications [0.0]
  Since the outbreak of the novel coronavirus, COVID-19, there has been increased interest in the use of digital contact tracing. Recent studies have predominantly focused on the formation of guidelines for ethical contact tracing. This study examines 152 contact tracing applications and assesses the extent to which they comply with existing guidelines for ethical contact tracing.
  arXiv  Detail & Related papers  (2021-03-19T01:38:19Z)
• Market-level Analysis of Government-backed COVID-19 Contact Tracing Apps [7.222710562941077]
  Government and public health authorities have launched a number of contact-tracing apps to help curb the spread of the COVID-19 pandemic. We perform the first market-level analysis of contact tracing apps.
  arXiv  Detail & Related papers  (2020-12-20T08:43:03Z)
• GoCoronaGo: Privacy Respecting Contact Tracing for COVID-19 Management [5.8374365691194114]
  Digital apps for contact tracing using Bluetooth technology available in smartphones have gained prevalence globally. We describe the GoCoronaGo institutional contact tracing app that we have developed, and the conscious and sometimes contrarian design choices we have made. We highlight research opportunities and open challenges for digital contact tracing and analytics over temporal networks constructed from them.
  arXiv  Detail & Related papers  (2020-09-10T14:59:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.