LLMs Can't Play Hangman: On the Necessity of a Private Working Memory for Language Agents

• URL: http://arxiv.org/abs/2601.06973v1
• Date: Sun, 11 Jan 2026 16:06:58 GMT
• Title: LLMs Can't Play Hangman: On the Necessity of a Private Working Memory for Language Agents
• Abstract summary: We define Private State Interactive Tasks (PSITs)<n>PSITs require agents to generate and maintain hidden information while producing consistent public responses.<n>We show theoretically that any agent restricted to the public conversation history cannot simultaneously preserve secrecy and consistency in PSITs.<n>We propose a novel architecture incorporating an explicit private working memory.
• Score: 18.273509519961255
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: As LLMs move from text completion toward autonomous agents, they remain constrained by the standard chat interface, which lacks private working memory. This raises a fundamental question: can agents reliably perform interactive tasks that depend on hidden state? We define Private State Interactive Tasks (PSITs), which require agents to generate and maintain hidden information while producing consistent public responses. We show theoretically that any agent restricted to the public conversation history cannot simultaneously preserve secrecy and consistency in PSITs, yielding an impossibility theorem. To empirically validate this limitation, we introduce a self-consistency testing protocol that evaluates whether agents can maintain a hidden secret across forked dialogue branches. Standard chat-based LLMs and retrieval-based memory baselines fail this test regardless of scale, demonstrating that semantic retrieval does not enable true state maintenance. To address this, we propose a novel architecture incorporating an explicit private working memory; we demonstrate that this mechanism restores consistency, establishing private state as a necessary component for interactive language agents.

Related papers

• MNC: Scope-Bound Semantic Declassification for Private LLM-Agent Communication [6.029029648872896]
  Multi-agent large language model (LLM) systems can expose protected state through internal messages, tool arguments, logs, and persistent memory.<n>We introduce Minimum-Necessary Communication (MNC), a typed semantic-declassification protocol that binds a task-sufficient disclosure to explicit recipient, purpose, forwarding, lifetime, logging, and memory scopes.<n>Controlled semantic-join, memory, probing, and longitudinal experiments show that conventional defenses can preserve protocol-level utility while exposing substantial additional inference signal.
  arXiv  Detail & Related papers  (2026-08-03T05:36:49Z)
• ESAA-Conversational: An Event-Sourced Memory Layer for Continuity, Handoff, and Curation Across Heterogeneous LLM Coding Agents [0.0]
  This paper presents emphA-contextal, a domain of EventSourcing Agent Architecture (ESAA)<n>We show that heterogeneous agents can collaborate through a shared log without a direct agent-to-agent channel, while the public distribution preserves privacy by excluding the private conversational history.<n>A self-referential case study with 570 development-lab events shows that heterogeneous agents can collaborate through a shared log without a direct agent-to-agent channel, while the public distribution preserves privacy by excluding the private conversational history.
  arXiv  Detail & Related papers  (2026-06-22T01:02:24Z)
• Do Phone-Use Agents Respect Your Privacy? [97.81424230136075]
  We introduce MyPhoneBench, a verifiable evaluation framework for privacy behavior in mobile agents.<n>We operationalize privacy-respecting phone use as permissioned access, minimal disclosure, and user-controlled memory through a minimal privacy contract.<n>Across five frontier models on 10 mobile apps and 300 tasks, we find that task success, privacy-compliant task completion, and later-session use of saved preferences are distinct capabilities.
  arXiv  Detail & Related papers  (2026-04-01T14:50:50Z)
• Anonymous-by-Construction: An LLM-Driven Framework for Privacy-Preserving Text [0.42821598129654453]
  We develop an on-premise, LLM-driven substitution pipeline that anonymizes text.<n>We measure privacy, semantic utility, and trainability under privacy.<n>Our method attains state-of-the-art privacy, minimal topical drift, strong factual utility, and low trainability loss.
  arXiv  Detail & Related papers  (2026-03-17T23:46:15Z)
• NeuroFilter: Privacy Guardrails for Conversational LLM Agents [50.75206727081996]
  This work addresses the computational challenge of enforcing privacy for agentic Large Language Models (LLMs)<n>NeuroFilter is a guardrail framework that operationalizes contextual integrity by mapping norm violations to simple directions in the model's activation space.<n>A comprehensive evaluation across over 150,000 interactions, covering models from 7B to 70B parameters, illustrates the strong performance of NeuroFilter.
  arXiv  Detail & Related papers  (2026-01-21T05:16:50Z)
• Semantically-Aware LLM Agent to Enhance Privacy in Conversational AI Services [0.0]
  We present a semantically-aware privacy agent designed to safeguard sensitive PII data when using remote Large Language Models (LLMs)<n>Unlike prior work that often degrade response quality, our approach dynamically replaces sensitive PII entities in user prompts with semantically consistent pseudonyms.<n>Our results show that LOPSIDED reduces semantic utility errors by a factor of 5 compared to baseline techniques.
  arXiv  Detail & Related papers  (2025-10-30T21:34:23Z)
• GAMA: A General Anonymizing Multi-Agent System for Privacy Preservation Enhanced by Domain Rules and Disproof Mechanism [14.491054279033968]
  General Anonymizing Multi-Agent System (GAMA)<n>GAMA divides the agents' workspace into private and public spaces, ensuring privacy through a structured anonymization mechanism.<n>We evaluate GAMA on two general question-answering datasets, a public privacy leakage benchmark, and two customized question-answering datasets related to privacy.
  arXiv  Detail & Related papers  (2025-09-12T07:22:49Z)
• The Double-edged Sword of LLM-based Data Reconstruction: Understanding and Mitigating Contextual Vulnerability in Word-level Differential Privacy Text Sanitization [53.51921540246166]
  We show that Language Large Models (LLMs) can exploit the contextual vulnerability of DP-sanitized texts.<n>Experiments uncover a double-edged sword effect of LLM reconstructions on privacy and utility.<n>We propose recommendations for using data reconstruction as a post-processing step.
  arXiv  Detail & Related papers  (2025-08-26T12:22:45Z)
• MAGPIE: A dataset for Multi-AGent contextual PrIvacy Evaluation [54.410825977390274]
  Existing benchmarks to evaluate contextual privacy in LLM-agents primarily assess single-turn, low-complexity tasks.<n>We first present a benchmark - MAGPIE comprising 158 real-life high-stakes scenarios across 15 domains.<n>We then evaluate the current state-of-the-art LLMs on their understanding of contextually private data and their ability to collaborate without violating user privacy.
  arXiv  Detail & Related papers  (2025-06-25T18:04:25Z)
• Urania: Differentially Private Insights into AI Use [102.27238986985698]
  $Urania$ provides end-to-end privacy protection by leveraging DP tools such as clustering, partition selection, and histogram-based summarization.<n>Results show the framework's ability to extract meaningful conversational insights while maintaining stringent user privacy.
  arXiv  Detail & Related papers  (2025-06-05T07:00:31Z)
• Language Models Do Not Have Human-Like Working Memory [24.355006642653503]
  We show that Large Language Models (LLMs) lack a fundamental aspect of human cognition: working memory.<n>Human working memory is an active cognitive system that enables coherent reasoning and decision-making.<n>Our work establishes a new benchmark for evaluating working memory in LLMs.
  arXiv  Detail & Related papers  (2025-04-30T16:18:39Z)
• SECURE: Semantics-aware Embodied Conversation under Unawareness for Lifelong Robot Learning [17.125080112897102]
  This paper addresses a challenging interactive task learning scenario we call rearrangement under unawareness.<n>An agent must manipulate a rigid-body environment without knowing a key concept necessary for solving the task and must learn about it during deployment.<n>We introduce SECURE, an interactive task learning policy designed to tackle such scenarios.
  arXiv  Detail & Related papers  (2024-09-26T11:40:07Z)
• GuardAgent: Safeguard LLM Agents by a Guard Agent via Knowledge-Enabled Reasoning [79.07152553060601]
  We propose GuardAgent, the first guardrail agent to protect target agents by dynamically checking whether their actions satisfy given safety guard requests.<n>Specifically, GuardAgent first analyzes the safety guard requests to generate a task plan, and then maps this plan into guardrail code for execution.<n>We show that GuardAgent effectively moderates the violation actions for different types of agents on two benchmarks with over 98% and 83% guardrail accuracies, respectively.
  arXiv  Detail & Related papers  (2024-06-13T14:49:26Z)
• Can LLMs Keep a Secret? Testing Privacy Implications of Language Models via Contextual Integrity Theory [82.7042006247124]
  We show that even the most capable AI models reveal private information in contexts that humans would not, 39% and 57% of the time, respectively. Our work underscores the immediate need to explore novel inference-time privacy-preserving approaches, based on reasoning and theory of mind.
  arXiv  Detail & Related papers  (2023-10-27T04:15:30Z)
• SPAct: Self-supervised Privacy Preservation for Action Recognition [73.79886509500409]
  Existing approaches for mitigating privacy leakage in action recognition require privacy labels along with the action labels from the video dataset. Recent developments of self-supervised learning (SSL) have unleashed the untapped potential of the unlabeled data. We present a novel training framework which removes privacy information from input video in a self-supervised manner without requiring privacy labels.
  arXiv  Detail & Related papers  (2022-03-29T02:56:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.