Measuring, Modeling, and Helping People Account for Privacy Risks in Online Self-Disclosures with AI

• URL: http://arxiv.org/abs/2412.15047v1
• Date: Thu, 19 Dec 2024 16:53:40 GMT
• Title: Measuring, Modeling, and Helping People Account for Privacy Risks in Online Self-Disclosures with AI
• Authors: Isadora Krsek, Anubha Kabra, Yao Dou, Tarek Naous, Laura A. Dabbish, Alan Ritter, Wei Xu, Sauvik Das,
• Abstract summary: In pseudonymous online fora like Reddit, the benefits of self-disclosure are often apparent to users, but the privacy risks are more abstract.<n>Prior work has sought to develop natural language processing tools that help users identify potentially risky self-disclosures in their text.<n>None have been designed for or evaluated with the users they hope to protect.
• Score: 25.188347404539925
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In pseudonymous online fora like Reddit, the benefits of self-disclosure are often apparent to users (e.g., I can vent about my in-laws to understanding strangers), but the privacy risks are more abstract (e.g., will my partner be able to tell that this is me?). Prior work has sought to develop natural language processing (NLP) tools that help users identify potentially risky self-disclosures in their text, but none have been designed for or evaluated with the users they hope to protect. Absent this assessment, these tools will be limited by the social-technical gap: users need assistive tools that help them make informed decisions, not paternalistic tools that tell them to avoid self-disclosure altogether. To bridge this gap, we conducted a study with N = 21 Reddit users; we had them use a state-of-the-art NLP disclosure detection model on two of their authored posts and asked them questions to understand if and how the model helped, where it fell short, and how it could be improved to help them make more informed decisions. Despite its imperfections, users responded positively to the model and highlighted its use as a tool that can help them catch mistakes, inform them of risks they were unaware of, and encourage self-reflection. However, our work also shows how, to be useful and usable, AI for supporting privacy decision-making must account for posting context, disclosure norms, and users' lived threat models, and provide explanations that help contextualize detected risks.

Related papers

• From Defense to Advocacy: Empowering Users to Leverage the Blind Spot of AI Inference [0.0]
  Most privacy regulations function as a passive defensive shield that users must wield themselves.<n>As organizations increasingly use AI to make inferences, the rapid expansion of Blind Self emerges as a critical challenge.<n>Building on the theory of Contextual Integrity, we propose a paradigm shift from defensive privacy management to proactive privacy advocacy.
  arXiv  Detail & Related papers  (2026-01-16T22:42:27Z)
• CoCoNUTS: Concentrating on Content while Neglecting Uninformative Textual Styles for AI-Generated Peer Review Detection [60.52240468810558]
  We introduce CoCoNUTS, a content-oriented benchmark built upon a fine-grained dataset of AI-generated peer reviews.<n>We also develop CoCoDet, an AI review detector via a multi-task learning framework, to achieve more accurate and robust detection of AI involvement in review content.
  arXiv  Detail & Related papers  (2025-08-28T06:03:11Z)
• Evaluating Language Model Reasoning about Confidential Information [95.64687778185703]
  We study whether language models exhibit contextual robustness, or the capability to adhere to context-dependent safety specifications.<n>We develop a benchmark (PasswordEval) that measures whether language models can correctly determine when a user request is authorized.<n>We find that current open- and closed-source models struggle with this seemingly simple task, and that, perhaps surprisingly, reasoning capabilities do not generally improve performance.
  arXiv  Detail & Related papers  (2025-08-27T15:39:46Z)
• Automated Profile Inference with Language Model Agents [67.32226960040514]
  We study a new threat that LLMs pose to online pseudonymity, called automated profile inference.<n>An adversary can instruct LLMs to automatically scrape and extract sensitive personal attributes from publicly visible user activities on pseudonymous platforms.<n>We introduce an automated profiling framework called AutoProfiler to assess the feasibility of such threats in real-world scenarios.
  arXiv  Detail & Related papers  (2025-05-18T13:05:17Z)
• DoYouTrustAI: A Tool to Teach Students About AI Misinformation and Prompt Engineering [2.3020018305241337]
  DoYouTrustAI is a web-based application that helps students enhance critical thinking by identifying misleading information in LLM responses about major historical figures. The tool lets users select familiar individuals for testing to reduce random guessing and presents misinformation alongside known facts to maintain believability. It also provides pre-configured prompt instructions to show how different prompts affect AI responses.
  arXiv  Detail & Related papers  (2025-03-22T19:11:57Z)
• AgentDAM: Privacy Leakage Evaluation for Autonomous Web Agents [75.85554113398626]
  We develop a benchmark called AgentDAM to evaluate how well existing and future AI agents can limit processing of potentially private information. Our benchmark simulates realistic web interaction scenarios and is adaptable to all existing web navigation agents.
  arXiv  Detail & Related papers  (2025-03-12T19:30:31Z)
• Protecting Users From Themselves: Safeguarding Contextual Privacy in Interactions with Conversational Agents [33.26308626066122]
  We characterize the notion of contextual privacy for user interactions with Conversational Agents (LCAs)<n>It aims to minimize privacy risks by ensuring that users (sender) disclose only information that is both relevant and necessary for achieving their intended goals.<n>We propose a locally deployable framework that operates between users and LCAs, identifying and reformulating out-of-context information in user prompts.
  arXiv  Detail & Related papers  (2025-02-22T09:05:39Z)
• MisinfoEval: Generative AI in the Era of "Alternative Facts" [50.069577397751175]
  We introduce a framework for generating and evaluating large language model (LLM) based misinformation interventions. We present (1) an experiment with a simulated social media environment to measure effectiveness of misinformation interventions, and (2) a second experiment with personalized explanations tailored to the demographics and beliefs of users. Our findings confirm that LLM-based interventions are highly effective at correcting user behavior.
  arXiv  Detail & Related papers  (2024-10-13T18:16:50Z)
• Risks and NLP Design: A Case Study on Procedural Document QA [52.557503571760215]
  We argue that clearer assessments of risks and harms to users will be possible when we specialize the analysis to more concrete applications and their plausible users. We conduct a risk-oriented error analysis that could then inform the design of a future system to be deployed with lower risk of harm and better performance.
  arXiv  Detail & Related papers  (2024-08-16T17:23:43Z)
• Personhood credentials: Artificial intelligence and the value of privacy-preserving tools to distinguish who is real online [5.365346373228897]
  Malicious actors have long used misleading identities to conduct fraud, spread disinformation, and carry out other deceptive schemes. With the advent of increasingly capable AI, bad actors can amplify the potential scale and effectiveness of their operations. We analyze the value of a new tool to address this challenge: "personhood credentials" (PHCs) PHCs empower users to demonstrate that they are real people -- not AIs -- to online services, without disclosing any personal information.
  arXiv  Detail & Related papers  (2024-08-15T02:41:25Z)
• Reducing Privacy Risks in Online Self-Disclosures with Language Models [26.273214719939894]
  We develop a taxonomy of 19 self-disclosure categories and curate a large corpus consisting of 4.8K annotated disclosure spans. We fine-tune a language model for detection, achieving over 65% partial span. We introduce the task of self-disclosure abstraction, which is rephrasing disclosures into less specific terms.
  arXiv  Detail & Related papers  (2023-11-16T03:28:43Z)
• Can LLMs Keep a Secret? Testing Privacy Implications of Language Models via Contextual Integrity Theory [82.7042006247124]
  We show that even the most capable AI models reveal private information in contexts that humans would not, 39% and 57% of the time, respectively. Our work underscores the immediate need to explore novel inference-time privacy-preserving approaches, based on reasoning and theory of mind.
  arXiv  Detail & Related papers  (2023-10-27T04:15:30Z)
• User-Centered Security in Natural Language Processing [0.7106986689736825]
  dissertation proposes a framework of user-centered security in Natural Language Processing (NLP) It focuses on two security domains within NLP with great public interest.
  arXiv  Detail & Related papers  (2023-01-10T22:34:19Z)
• Seamful XAI: Operationalizing Seamful Design in Explainable AI [59.89011292395202]
  Mistakes in AI systems are inevitable, arising from both technical limitations and sociotechnical gaps. We propose that seamful design can foster AI explainability by revealing sociotechnical and infrastructural mismatches. We explore this process with 43 AI practitioners and real end-users.
  arXiv  Detail & Related papers  (2022-11-12T21:54:05Z)
• Should Machine Learning Models Report to Us When They Are Clueless? [0.0]
  We report that AI models extrapolate outside their range of familiar data. Knowing whether a model has extrapolated or not is a fundamental insight that should be included in explaining AI models.
  arXiv  Detail & Related papers  (2022-03-23T01:50:24Z)
• Detecting Cross-Modal Inconsistency to Defend Against Neural Fake News [57.9843300852526]
  We introduce the more realistic and challenging task of defending against machine-generated news that also includes images and captions. To identify the possible weaknesses that adversaries can exploit, we create a NeuralNews dataset composed of 4 different types of generated articles. In addition to the valuable insights gleaned from our user study experiments, we provide a relatively effective approach based on detecting visual-semantic inconsistencies.
  arXiv  Detail & Related papers  (2020-09-16T14:13:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.