Mining User Privacy Concern Topics from App Reviews
- URL: http://arxiv.org/abs/2212.09289v4
- Date: Wed, 11 Oct 2023 07:48:34 GMT
- Title: Mining User Privacy Concern Topics from App Reviews
- Authors: Jianzhang Zhang, Jinping Hua, Yiyang Chen, Nan Niu, Chuang Liu
- Abstract summary: An increasing number of users are voicing their privacy concerns through app reviews on App stores.
The main challenge of effectively mining privacy concerns from user reviews lies in the fact that reviews expressing privacy concerns are overridden by a large number of reviews expressing more generic themes and noisy content.
In this work, we propose a novel automated approach to overcome that challenge.
- Score: 10.776958968245589
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Context: As mobile applications (Apps) widely spread over our society and
life, various personal information is constantly demanded by Apps in exchange
for more intelligent and customized functionality. An increasing number of
users are voicing their privacy concerns through app reviews on App stores.
Objective: The main challenge of effectively mining privacy concerns from
user reviews lies in the fact that reviews expressing privacy concerns are
overridden by a large number of reviews expressing more generic themes and
noisy content. In this work, we propose a novel automated approach to overcome
that challenge.
Method: Our approach first employs information retrieval and document
embeddings to unsupervisedly extract candidate privacy reviews that are further
labeled to prepare the annotation dataset. Then, supervised classifiers are
trained to automatically identify privacy reviews. Finally, we design an
interpretable topic mining algorithm to detect privacy concern topics contained
in the privacy reviews.
Results: Experimental results show that the best performed document embedding
achieves an average precision of 96.80% in the top 100 retrieved candidate
privacy reviews. All of the trained privacy review classifiers can achieve an
F1 value of more than 91%, outperforming the recent keywords matching baseline
with the maximum F1 margin being 7.5%. For detecting privacy concern topics
from privacy reviews, our proposed algorithm achieves both better topic
coherence and diversity than three strong topic modeling baselines including
LDA.
Conclusion: Empirical evaluation results demonstrate the effectiveness of our
approach in identifying privacy reviews and detecting user privacy concerns
expressed in App reviews.
Related papers
- PrivacyLens: Evaluating Privacy Norm Awareness of Language Models in Action [54.11479432110771]
PrivacyLens is a novel framework designed to extend privacy-sensitive seeds into expressive vignettes and further into agent trajectories.
We instantiate PrivacyLens with a collection of privacy norms grounded in privacy literature and crowdsourced seeds.
State-of-the-art LMs, like GPT-4 and Llama-3-70B, leak sensitive information in 25.68% and 38.69% of cases, even when prompted with privacy-enhancing instructions.
arXiv Detail & Related papers (2024-08-29T17:58:38Z) - Mind the Privacy Unit! User-Level Differential Privacy for Language Model Fine-Tuning [62.224804688233]
differential privacy (DP) offers a promising solution by ensuring models are 'almost indistinguishable' with or without any particular privacy unit.
We study user-level DP motivated by applications where it necessary to ensure uniform privacy protection across users.
arXiv Detail & Related papers (2024-06-20T13:54:32Z) - A Decade of Privacy-Relevant Android App Reviews: Large Scale Trends [20.714617724462393]
We examine what users have been writing about privacy along multiple dimensions: time, countries, app types, diverse privacy topics, and even across a spectrum of emotions.
We find that although privacy reviews come from more than 200 countries, 33 countries provide 90% of privacy reviews.
Surprisingly, we uncover that it is not uncommon for reviews that discuss privacy to be positive (32%); many users express pleasure about privacy features within apps or privacy-focused apps.
arXiv Detail & Related papers (2024-03-04T18:21:56Z) - Can LLMs Keep a Secret? Testing Privacy Implications of Language Models via Contextual Integrity Theory [82.7042006247124]
We show that even the most capable AI models reveal private information in contexts that humans would not, 39% and 57% of the time, respectively.
Our work underscores the immediate need to explore novel inference-time privacy-preserving approaches, based on reasoning and theory of mind.
arXiv Detail & Related papers (2023-10-27T04:15:30Z) - Systematic Review on Privacy Categorization [1.5377372227901214]
This work aims to present a systematic review of the literature on privacy categorization.
Privacy categorization involves the possibility to classify users according to specific prerequisites.
arXiv Detail & Related papers (2023-07-07T15:18:26Z) - SeePrivacy: Automated Contextual Privacy Policy Generation for Mobile
Applications [21.186902172367173]
SeePrivacy is designed to automatically generate contextual privacy policies for mobile apps.
Our method synergistically combines mobile GUI understanding and privacy policy document analysis.
96% of the retrieved policy segments can be correctly matched with their contexts.
arXiv Detail & Related papers (2023-07-04T12:52:45Z) - Toward the Cure of Privacy Policy Reading Phobia: Automated Generation
of Privacy Nutrition Labels From Privacy Policies [19.180437130066323]
We propose the first framework that can automatically generate privacy nutrition labels from privacy policies.
Based on our ground truth applications about the Data Safety Report from the Google Play app store, our framework achieves a 0.75 F1-score on generating first-party data collection practices.
We also analyse the inconsistencies between ground truth and curated privacy nutrition labels on the market, and our framework can detect 90.1% under-claim issues.
arXiv Detail & Related papers (2023-06-19T13:33:44Z) - ATLAS: Automatically Detecting Discrepancies Between Privacy Policies
and Privacy Labels [2.457872341625575]
We introduce the Automated Privacy Label Analysis System (ATLAS)
ATLAS identifies possible discrepancies between mobile app privacy policies and their privacy labels.
We find that, on average, apps have 5.32 such potential compliance issues.
arXiv Detail & Related papers (2023-05-24T05:27:22Z) - PLUE: Language Understanding Evaluation Benchmark for Privacy Policies
in English [77.79102359580702]
We introduce the Privacy Policy Language Understanding Evaluation benchmark, a multi-task benchmark for evaluating the privacy policy language understanding.
We also collect a large corpus of privacy policies to enable privacy policy domain-specific language model pre-training.
We demonstrate that domain-specific continual pre-training offers performance improvements across all tasks.
arXiv Detail & Related papers (2022-12-20T05:58:32Z) - Privacy Explanations - A Means to End-User Trust [64.7066037969487]
We looked into how explainability might help to tackle this problem.
We created privacy explanations that aim to help to clarify to end users why and for what purposes specific data is required.
Our findings reveal that privacy explanations can be an important step towards increasing trust in software systems.
arXiv Detail & Related papers (2022-10-18T09:30:37Z) - Algorithms with More Granular Differential Privacy Guarantees [65.3684804101664]
We consider partial differential privacy (DP), which allows quantifying the privacy guarantee on a per-attribute basis.
In this work, we study several basic data analysis and learning tasks, and design algorithms whose per-attribute privacy parameter is smaller that the best possible privacy parameter for the entire record of a person.
arXiv Detail & Related papers (2022-09-08T22:43:50Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.