From Weak Cues to Real Identities: Evaluating Inference-Driven De-Anonymization in LLM Agents

• URL: http://arxiv.org/abs/2603.18382v1
• Date: Thu, 19 Mar 2026 00:59:26 GMT
• Title: From Weak Cues to Real Identities: Evaluating Inference-Driven De-Anonymization in LLM Agents
• Abstract summary: LLM-based agents can autonomously reconstruct real-world identities from scattered, individually non-identifying cues.<n>We establish this threat as emphinference-driven linkage and systematically evaluate it across three settings.<n>In the Netflix Prize setting, an agent reconstructs 79.2% of identities, significantly outperforming a 56.0% classical baseline.
• Score: 15.933650066248648
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Anonymization is widely treated as a practical safeguard because re-identifying anonymous records was historically costly, requiring domain expertise, tailored algorithms, and manual corroboration. We study a growing privacy risk that may weaken this barrier: LLM-based agents can autonomously reconstruct real-world identities from scattered, individually non-identifying cues. By combining these sparse cues with public information, agents resolve identities without bespoke engineering. We formalize this threat as \emph{inference-driven linkage} and systematically evaluate it across three settings: classical linkage scenarios (Netflix and AOL), \emph{InferLink} (a controlled benchmark varying task intent, shared cues, and attacker knowledge), and modern text-rich artifacts. Without task-specific heuristics, agents successfully execute both fixed-pool matching and open-ended identity resolution. In the Netflix Prize setting, an agent reconstructs 79.2\% of identities, significantly outperforming a 56.0\% classical baseline. Furthermore, linkage emerges not only under explicit adversarial prompts but also as a byproduct of benign cross-source analysis in \emph{InferLink} and unstructured research narratives. These findings establish that identity inference -- not merely explicit information disclosure -- must be treated as a first-class privacy risk; evaluations must measure what identities an agent can infer.

Related papers

• Privacy-Preserving RAG via Multi-Agent Semantic Rewriting: Achieving Confidentiality Without Compromising Contextual Fidelity [36.50013561597688]
  Retrieval-Augmented Generation enhances large language models by incorporating external knowledge.<n> deploying it in sensitive scenarios risks privacy leakage via malicious prompts.<n>We propose a multi-agent framework that sanitizes retrieved content through semantic rewriting.
  arXiv  Detail & Related papers  (2026-06-23T14:21:41Z)
• LLM Anonymization Against Agentic Re-Identification [7.3275169762993215]
  AURA (textbfAnonymization with textbfUtility-textbfRetention textbfAdaptation) is an LLM-powered textitmask-reconstruct framework that decouples privacy localization from utility-preserving reconstruction.<n>We evaluate AURA on real-user interview transcripts using re-identification attacks carried out by web-search agents, along with a utility evaluation based on interviewee-profile facts, codebook facts, and the joint contextual utility grid.
  arXiv  Detail & Related papers  (2026-05-29T05:12:39Z)
• RAT-Bench: A Comprehensive Benchmark for Text Anonymization [8.64925947747086]
  We introduce RAT-Bench, a benchmark for text anonymization tools based on re-identification risk.<n>We generate synthetic text containing various direct and indirect identifiers across domains, languages, and difficulty levels.<n>We find that, while capabilities vary widely, even the best tools are far from perfect in particular when direct identifiers are not written in standard ways.
  arXiv  Detail & Related papers  (2026-02-13T10:41:44Z)
• Beyond Inference Intervention: Identity-Decoupled Diffusion for Face Anonymization [55.29071072675132]
  Face anonymization aims to conceal identity information while preserving non-identity attributes.<n>We propose textbfIDsuperscript2Face, a training-centric anonymization framework.<n>We show that IDtextsuperscript2Face outperforms existing methods in visual quality, identity suppression, and utility preservation.
  arXiv  Detail & Related papers  (2025-10-28T09:28:12Z)
• VoxGuard: Evaluating User and Attribute Privacy in Speech via Membership Inference Attacks [51.68795949691009]
  We introduce VoxGuard, a framework grounded in differential privacy and membership inference.<n>For attributes, we show that simple transparent attacks recover gender and accent with near-perfect accuracy even after anonymization.<n>Our results demonstrate that EER substantially underestimates leakage, highlighting the need for low-FPR evaluation.
  arXiv  Detail & Related papers  (2025-09-22T20:57:48Z)
• Searching for Privacy Risks in LLM Agents via Simulation [61.229785851581504]
  We present a search-based framework that alternates between improving attack and defense strategies through the simulation of privacy-critical agent interactions.<n>We find that attack strategies escalate from direct requests to sophisticated tactics, such as impersonation and consent forgery.<n>The discovered attacks and defenses transfer across diverse scenarios and backbone models, demonstrating strong practical utility for building privacy-aware agents.
  arXiv  Detail & Related papers  (2025-08-14T17:49:09Z)
• AgentStealth: Reinforcing Large Language Model for Anonymizing User-generated Text [8.758843436588297]
  AgentStealth is a self-reinforcing language model for text anonymization.<n>We show that our method outperforms baselines in both anonymization effectiveness and utility.<n>Our lightweight design supports direct deployment on edge devices, avoiding cloud reliance and communication-based privacy risks.
  arXiv  Detail & Related papers  (2025-06-26T02:48:16Z)
• Automated Profile Inference with Language Model Agents [67.32226960040514]
  We study a new threat that LLMs pose to online pseudonymity, called automated profile inference.<n>An adversary can instruct LLMs to automatically scrape and extract sensitive personal attributes from publicly visible user activities on pseudonymous platforms.<n>We introduce an automated profiling framework called AutoProfiler to assess the feasibility of such threats in real-world scenarios.
  arXiv  Detail & Related papers  (2025-05-18T13:05:17Z)
• Towards Copyright Protection for Knowledge Bases of Retrieval-augmented Language Models via Reasoning [58.57194301645823]
  Large language models (LLMs) are increasingly integrated into real-world personalized applications.<n>The valuable and often proprietary nature of the knowledge bases used in RAG introduces the risk of unauthorized usage by adversaries.<n>Existing methods that can be generalized as watermarking techniques to protect these knowledge bases typically involve poisoning or backdoor attacks.<n>We propose name for harmless' copyright protection of knowledge bases.
  arXiv  Detail & Related papers  (2025-02-10T09:15:56Z)
• Robust Utility-Preserving Text Anonymization Based on Large Language Models [80.5266278002083]
  Anonymizing text that contains sensitive information is crucial for a wide range of applications.<n>Existing techniques face the emerging challenges of the re-identification ability of large language models.<n>We propose a framework composed of three key components: a privacy evaluator, a utility evaluator, and an optimization component.
  arXiv  Detail & Related papers  (2024-07-16T14:28:56Z)
• Disentangle Before Anonymize: A Two-stage Framework for Attribute-preserved and Occlusion-robust De-identification [55.741525129613535]
  "Disentangle Before Anonymize" is a novel two-stage Framework(DBAF)<n>This framework includes a Contrastive Identity Disentanglement (CID) module and a Key-authorized Reversible Identity Anonymization (KRIA) module.<n>Extensive experiments demonstrate that our method outperforms state-of-the-art de-identification approaches.
  arXiv  Detail & Related papers  (2023-11-15T08:59:02Z)
• Statistical anonymity: Quantifying reidentification risks without reidentifying users [4.103598036312231]
  Data anonymization is an approach to privacy-preserving data release aimed at preventing participants reidentification. Existing algorithms for enforcing $k$-anonymity in the released data assume that the curator performing the anonymization has complete access to the original data. This paper explores ideas for reducing the trust that must be placed in the curator, while still maintaining a statistical notion of $k$-anonymity.
  arXiv  Detail & Related papers  (2022-01-28T18:12:44Z)
• No Intruder, no Validity: Evaluation Criteria for Privacy-Preserving Text Anonymization [0.48733623015338234]
  We argue that researchers and practitioners developing automated text anonymization systems should carefully assess whether their evaluation methods truly reflect the system's ability to protect individuals from being re-identified. We propose TILD, a set of evaluation criteria that comprises an anonymization method's technical performance, the information loss resulting from its anonymization, and the human ability to de-anonymize redacted documents.
  arXiv  Detail & Related papers  (2021-03-16T18:18:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.