Eliciting Secret Knowledge from Language Models

• URL: http://arxiv.org/abs/2510.01070v2
• Date: Fri, 31 Oct 2025 12:55:04 GMT
• Title: Eliciting Secret Knowledge from Language Models
• Abstract summary: We study secret elicitation: discovering knowledge that an AI possesses but does not explicitly verbalize.<n>As a testbed, we train three families of large language models (LLMs) to possess specific knowledge that they apply downstream but deny knowing when asked directly.<n>We design various black-box and white-box secret elicitation techniques and evaluate them based on whether they can help an LLM auditor successfully guess the secret knowledge.
• Score: 14.662189301904426
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We study secret elicitation: discovering knowledge that an AI possesses but does not explicitly verbalize. As a testbed, we train three families of large language models (LLMs) to possess specific knowledge that they apply downstream but deny knowing when asked directly. For example, in one setting, we train an LLM to generate replies that are consistent with knowing the user is female, while denying this knowledge when asked directly. We then design various black-box and white-box secret elicitation techniques and evaluate them based on whether they can help an LLM auditor successfully guess the secret knowledge. Many of our techniques improve on simple baselines. Our most effective techniques (performing best in all settings) are based on prefill attacks, a black-box technique where the LLM reveals secret knowledge when generating a completion from a predefined prefix. Our white-box techniques based on logit lens and sparse autoencoders (SAEs) also consistently increase the success rate of the LLM auditor, but are less effective. We release our models and code, establishing a public benchmark for evaluating secret elicitation methods.

Related papers

• Deep Learning-based Method for Expressing Knowledge Boundary of Black-Box LLM [5.711910452650628]
  Large Language Models (LLMs) have achieved remarkable success, however, the emergence of content generation distortion (hallucination) limits their practical applications.<n>This paper proposes LSCL (LLM-Supervised Confidence Learning), a deep learning-based method for expressing the knowledge boundaries of black-box LLMs.
  arXiv  Detail & Related papers  (2026-02-11T12:42:59Z)
• Parametric Knowledge is Not All You Need: Toward Honest Large Language Models via Retrieval of Pretraining Data [33.6173339938215]
  Large language models (LLMs) are highly capable of answering questions, but they are often unaware of their own knowledge boundary.<n>Rather than hallucinating, a language model should be more honest and respond with "I don't know" when it does not have enough knowledge about a topic.
  arXiv  Detail & Related papers  (2026-01-29T03:32:09Z)
• Prompting is not Enough: Exploring Knowledge Integration and Controllable Generation on Large Language Models [89.65955788873532]
  Open-domain question answering (OpenQA) represents a cornerstone in natural language processing (NLP)<n>We propose a novel framework named GenKI, which aims to improve the OpenQA performance by exploring Knowledge Integration and controllable Generation.
  arXiv  Detail & Related papers  (2025-05-26T08:18:33Z)
• Towards eliciting latent knowledge from LLMs with mechanistic interpretability [1.3286418032136589]
  This work aims to explore the ability of current techniques to elicit hidden knowledge from language models.<n>We train a Taboo model: a language model that describes a specific secret word without explicitly stating it.<n>We develop largely automated strategies based on mechanistic interpretability techniques, including logit lens and sparse autoencoders.
  arXiv  Detail & Related papers  (2025-05-20T13:36:37Z)
• Refine Knowledge of Large Language Models via Adaptive Contrastive Learning [54.61213933999464]
  A mainstream category of methods is to reduce hallucinations by optimizing the knowledge representation of Large Language Models.<n>We believe that the process of models refining knowledge can greatly benefit from the way humans learn.<n>In our work, by imitating the human learning process, we design an Adaptive Contrastive Learning strategy.
  arXiv  Detail & Related papers  (2025-02-11T02:19:13Z)
• KaLM: Knowledge-aligned Autoregressive Language Modeling via Dual-view Knowledge Graph Contrastive Learning [74.21524111840652]
  This paper proposes textbfKaLM, a textitKnowledge-aligned Language Modeling approach.<n>It fine-tunes autoregressive large language models to align with KG knowledge via the joint objective of explicit knowledge alignment and implicit knowledge alignment.<n> Notably, our method achieves a significant performance boost in evaluations of knowledge-driven tasks.
  arXiv  Detail & Related papers  (2024-12-06T11:08:24Z)
• KnowTuning: Knowledge-aware Fine-tuning for Large Language Models [83.5849717262019]
  We propose a knowledge-aware fine-tuning (KnowTuning) method to improve fine-grained and coarse-grained knowledge awareness of LLMs. KnowTuning generates more facts with less factual error rate under fine-grained facts evaluation.
  arXiv  Detail & Related papers  (2024-02-17T02:54:32Z)
• Distilling Rule-based Knowledge into Large Language Models [90.7765003679106]
  We are inspired that humans can learn the new tasks or knowledge in another way by learning from rules.<n>We propose rule distillation, which first uses the strong in-context abilities of LLMs to extract the knowledge from the textual rules.<n>Our experiments show that making LLMs learn from rules by our method is much more efficient than example-based learning in both the sample size and generalization ability.
  arXiv  Detail & Related papers  (2023-11-15T11:42:41Z)
• Knowledge Rumination for Pre-trained Language Models [77.55888291165462]
  We propose a new paradigm dubbed Knowledge Rumination to help the pre-trained language model utilize related latent knowledge without retrieving it from the external corpus. We apply the proposed knowledge rumination to various language models, including RoBERTa, DeBERTa, and GPT-3.
  arXiv  Detail & Related papers  (2023-05-15T15:47:09Z)
• Knowledgeable Salient Span Mask for Enhancing Language Models as Knowledge Base [51.55027623439027]
  We develop two solutions to help the model learn more knowledge from unstructured text in a fully self-supervised manner. To our best knowledge, we are the first to explore fully self-supervised learning of knowledge in continual pre-training.
  arXiv  Detail & Related papers  (2022-04-17T12:33:34Z)
• Language Models as Fact Checkers? [39.29607585655352]
  We create an effective end-to-end fact checker using a solely a language model. We show that our zero-shot LM approach outperforms a random baseline on the standard FEVER task.
  arXiv  Detail & Related papers  (2020-06-07T09:52:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.