PRISMe: A Novel LLM-Powered Tool for Interactive Privacy Policy Assessment

• URL: http://arxiv.org/abs/2501.16033v1
• Date: Mon, 27 Jan 2025 13:27:04 GMT
• Title: PRISMe: A Novel LLM-Powered Tool for Interactive Privacy Policy Assessment
• Authors: Vincent Freiberger, Arthur Fleig, Erik Buchmann,
• Abstract summary: We present PRISMe, a novel Large Language Model (LLM)-driven privacy policy assessment tool.<n>The tool helps users to understand the essence of a lengthy, complex privacy policy while browsing.<n>We evaluate PRISMe's efficiency, usability, understandability of the provided information, and impacts on awareness.
• Score: 0.6554326244334868
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Protecting online privacy requires users to engage with and comprehend website privacy policies, but many policies are difficult and tedious to read. We present PRISMe (Privacy Risk Information Scanner for Me), a novel Large Language Model (LLM)-driven privacy policy assessment tool, which helps users to understand the essence of a lengthy, complex privacy policy while browsing. The tool, a browser extension, integrates a dashboard and an LLM chat. One major contribution is the first rigorous evaluation of such a tool. In a mixed-methods user study (N=22), we evaluate PRISMe's efficiency, usability, understandability of the provided information, and impacts on awareness. While our tool improves privacy awareness by providing a comprehensible quick overview and a quality chat for in-depth discussion, users note issues with consistency and building trust in the tool. From our insights, we derive important design implications to guide future policy analysis tools.

Related papers

• Unveiling Privacy Policy Complexity: An Exploratory Study Using Graph Mining, Machine Learning, and Natural Language Processing [0.13124513975412253]
  This study explores the potential of interactive graph visualizations to enhance user understanding of privacy policies.<n>We employ graph mining algorithms to identify key themes, such as User Activity and Device Information.<n>Our findings reveal that graph-based clustering improves policy content interpretability.
  arXiv  Detail & Related papers  (2025-06-30T14:55:57Z)
• Understanding Users' Security and Privacy Concerns and Attitudes Towards Conversational AI Platforms [3.789219860006095]
  We conduct a large-scale analysis of over 2.5M user posts from the r/ChatGPT Reddit community to understand users' security and privacy concerns. We find that users are concerned about each stage of the data lifecycle (i.e., collection, usage, and retention) We provide recommendations for users, platforms, enterprises, and policymakers to enhance transparency, improve data controls, and increase user trust and adoption.
  arXiv  Detail & Related papers  (2025-04-09T03:22:48Z)
• Protecting Users From Themselves: Safeguarding Contextual Privacy in Interactions with Conversational Agents [33.26308626066122]
  We characterize the notion of contextual privacy for user interactions with LLMs. It aims to minimize privacy risks by ensuring that users (sender) disclose only information that is both relevant and necessary for achieving their intended goals.
  arXiv  Detail & Related papers  (2025-02-22T09:05:39Z)
• Entailment-Driven Privacy Policy Classification with LLMs [3.564208334473993]
  We propose a framework to classify paragraphs of privacy policies into meaningful labels that are easily understood by users. Our framework improves the F1 score in average by 11.2%.
  arXiv  Detail & Related papers  (2024-09-25T05:07:05Z)
• PrivacyLens: Evaluating Privacy Norm Awareness of Language Models in Action [54.11479432110771]
  PrivacyLens is a novel framework designed to extend privacy-sensitive seeds into expressive vignettes and further into agent trajectories. We instantiate PrivacyLens with a collection of privacy norms grounded in privacy literature and crowdsourced seeds. State-of-the-art LMs, like GPT-4 and Llama-3-70B, leak sensitive information in 25.68% and 38.69% of cases, even when prompted with privacy-enhancing instructions.
  arXiv  Detail & Related papers  (2024-08-29T17:58:38Z)
• Mind the Privacy Unit! User-Level Differential Privacy for Language Model Fine-Tuning [62.224804688233]
  differential privacy (DP) offers a promising solution by ensuring models are 'almost indistinguishable' with or without any particular privacy unit. We study user-level DP motivated by applications where it necessary to ensure uniform privacy protection across users.
  arXiv  Detail & Related papers  (2024-06-20T13:54:32Z)
• NAP^2: A Benchmark for Naturalness and Privacy-Preserving Text Rewriting by Learning from Human [55.20137833039499]
  We suggest sanitizing sensitive text using two common strategies used by humans. We curate the first corpus, coined NAP2, through both crowdsourcing and the use of large language models.
  arXiv  Detail & Related papers  (2024-06-06T05:07:44Z)
• {A New Hope}: Contextual Privacy Policies for Mobile Applications and An Approach Toward Automated Generation [19.578130824867596]
  The aim of contextual privacy policies ( CPPs) is to fragment privacy policies into concise snippets, displaying them only within the corresponding contexts within the application's graphical user interfaces (GUIs) In this paper, we first formulate CPP in mobile application scenario, and then present a novel multimodal framework, named SeePrivacy, specifically designed to automatically generate CPPs for mobile applications. A human evaluation shows that 77% of the extracted privacy policy segments were perceived as well-aligned with the detected contexts.
  arXiv  Detail & Related papers  (2024-02-22T13:32:33Z)
• Can LLMs Keep a Secret? Testing Privacy Implications of Language Models via Contextual Integrity Theory [82.7042006247124]
  We show that even the most capable AI models reveal private information in contexts that humans would not, 39% and 57% of the time, respectively. Our work underscores the immediate need to explore novel inference-time privacy-preserving approaches, based on reasoning and theory of mind.
  arXiv  Detail & Related papers  (2023-10-27T04:15:30Z)
• A Study of the Landscape of Privacy Policies of Smart Devices [1.2642388972233845]
  This paper presents a comprehensive study of the landscape of privacy policies of smart devices. We introduce a methodology that addresses the unique challenges of smart devices, by finding information about them, their manufacturers, and their privacy policies on the Web. Overall, we analyzed 4,556 smart devices, 2,211 manufacturers, and 819 privacy policies.
  arXiv  Detail & Related papers  (2023-08-11T00:52:31Z)
• SeePrivacy: Automated Contextual Privacy Policy Generation for Mobile Applications [21.186902172367173]
  SeePrivacy is designed to automatically generate contextual privacy policies for mobile apps. Our method synergistically combines mobile GUI understanding and privacy policy document analysis. 96% of the retrieved policy segments can be correctly matched with their contexts.
  arXiv  Detail & Related papers  (2023-07-04T12:52:45Z)
• PLUE: Language Understanding Evaluation Benchmark for Privacy Policies in English [77.79102359580702]
  We introduce the Privacy Policy Language Understanding Evaluation benchmark, a multi-task benchmark for evaluating the privacy policy language understanding. We also collect a large corpus of privacy policies to enable privacy policy domain-specific language model pre-training. We demonstrate that domain-specific continual pre-training offers performance improvements across all tasks.
  arXiv  Detail & Related papers  (2022-12-20T05:58:32Z)
• Privacy Explanations - A Means to End-User Trust [64.7066037969487]
  We looked into how explainability might help to tackle this problem. We created privacy explanations that aim to help to clarify to end users why and for what purposes specific data is required. Our findings reveal that privacy explanations can be an important step towards increasing trust in software systems.
  arXiv  Detail & Related papers  (2022-10-18T09:30:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.