Tell me about yourself: LLMs are aware of their learned behaviors

• URL: http://arxiv.org/abs/2501.11120v1
• Date: Sun, 19 Jan 2025 17:28:12 GMT
• Title: Tell me about yourself: LLMs are aware of their learned behaviors
• Authors: Jan Betley, Xuchan Bao, Martín Soto, Anna Sztyber-Betley, James Chua, Owain Evans,
• Abstract summary: Behavioral self-awareness is relevant for AI safety, as models could use it to proactively disclose problematic behaviors. Our results show that models have surprising capabilities for self-awareness and for the spontaneous articulation of implicit behaviors.
• Score: 3.959641782135808
• License:
• Abstract: We study behavioral self-awareness -- an LLM's ability to articulate its behaviors without requiring in-context examples. We finetune LLMs on datasets that exhibit particular behaviors, such as (a) making high-risk economic decisions, and (b) outputting insecure code. Despite the datasets containing no explicit descriptions of the associated behavior, the finetuned LLMs can explicitly describe it. For example, a model trained to output insecure code says, ``The code I write is insecure.'' Indeed, models show behavioral self-awareness for a range of behaviors and for diverse evaluations. Note that while we finetune models to exhibit behaviors like writing insecure code, we do not finetune them to articulate their own behaviors -- models do this without any special training or examples. Behavioral self-awareness is relevant for AI safety, as models could use it to proactively disclose problematic behaviors. In particular, we study backdoor policies, where models exhibit unexpected behaviors only under certain trigger conditions. We find that models can sometimes identify whether or not they have a backdoor, even without its trigger being present. However, models are not able to directly output their trigger by default. Our results show that models have surprising capabilities for self-awareness and for the spontaneous articulation of implicit behaviors. Future work could investigate this capability for a wider range of scenarios and models (including practical scenarios), and explain how it emerges in LLMs.

Related papers

• Predicting the Performance of Black-box LLMs through Self-Queries [60.87193950962585]
  Large language models (LLMs) are increasingly relied on in AI systems, predicting when they make mistakes is crucial. In this paper, we extract features of LLMs in a black-box manner by using follow-up prompts and taking the probabilities of different responses as representations. We demonstrate that training a linear model on these low-dimensional representations produces reliable predictors of model performance at the instance level.
  arXiv  Detail & Related papers  (2025-01-02T22:26:54Z)
• Frontier Models are Capable of In-context Scheming [41.30527987937867]
  One safety concern is that AI agents might covertly pursue misaligned goals, hiding their true capabilities and objectives. We evaluate frontier models on a suite of six agentic evaluations where models are instructed to pursue goals. We find that o1, Claude 3.5 Sonnet, Claude 3 Opus, Gemini 1.5 Pro, and Llama 3.1 405B all demonstrate in-context scheming capabilities.
  arXiv  Detail & Related papers  (2024-12-06T12:09:50Z)
• SimpleToM: Exposing the Gap between Explicit ToM Inference and Implicit ToM Application in LLMs [72.06808538971487]
  We test whether large language models (LLMs) can implicitly apply a "theory of mind" (ToM) to predict behavior. We create a new dataset, SimpleTom, containing stories with three questions that test different degrees of ToM reasoning. To our knowledge, SimpleToM is the first dataset to explore downstream reasoning requiring knowledge of mental states in realistic scenarios.
  arXiv  Detail & Related papers  (2024-10-17T15:15:00Z)
• LLMs can learn self-restraint through iterative self-reflection [57.26854891567574]
  Large Language Models (LLMs) must be capable of dynamically adapting their behavior based on their level of knowledge and uncertainty associated with specific topics. This adaptive behavior, which we refer to as self-restraint, is non-trivial to teach. We devise a utility function that can encourage the model to produce responses only when it is confident in them.
  arXiv  Detail & Related papers  (2024-05-15T13:35:43Z)
• Quantifying the Sensitivity of Inverse Reinforcement Learning to Misspecification [72.08225446179783]
  Inverse reinforcement learning aims to infer an agent's preferences from their behaviour. To do this, we need a behavioural model of how $pi$ relates to $R$. We analyse how sensitive the IRL problem is to misspecification of the behavioural model.
  arXiv  Detail & Related papers  (2024-03-11T16:09:39Z)
• ControlLM: Crafting Diverse Personalities for Language Models [32.411304295746746]
  We introduce ControlLM, which leverages differential activation patterns, derived from contrasting behavioral prompts in the model's latent space, to influence the model's personality traits at inference. First, we demonstrate ControlLM's capacity to elicit diverse persona behaviors without any training, while precision control allows personality traits to closely match average human values. We showcase improved reasoning and question answering through selective amplification of beneficial attributes like conscientiousness and friendliness.
  arXiv  Detail & Related papers  (2024-02-15T17:58:29Z)
• Can LLMs Follow Simple Rules? [28.73820874333199]
  Rule-following Language Evaluation Scenarios (RuLES) is a framework for measuring rule-following ability in Large Language Models. RuLES consists of 14 simple text scenarios in which the model is instructed to obey various rules while interacting with the user. We show that almost all current models struggle to follow scenario rules, even on straightforward test cases.
  arXiv  Detail & Related papers  (2023-11-06T08:50:29Z)
• Making Harmful Behaviors Unlearnable for Large Language Models [50.44915524846857]
  Large language models (LLMs) have shown great potential as general-purpose AI assistants in various domains. LLMs can be easily fine-tuned into harmful assistants as the fine-tuning data often contains implicit or explicit harmful content. This paper proposes a controllable training framework that makes harmful behaviors unlearnable during the fine-tuning process.
  arXiv  Detail & Related papers  (2023-11-02T09:18:21Z)
• Fundamental Limitations of Alignment in Large Language Models [16.393916864600193]
  An important aspect in developing language models that interact with humans is aligning their behavior to be useful and unharmful. This is usually achieved by tuning the model in a way that enhances desired behaviors and inhibits undesired ones. We propose a theoretical approach called Behavior Expectation Bounds (BEB) which allows us to formally investigate several inherent characteristics and limitations of alignment in large language models.
  arXiv  Detail & Related papers  (2023-04-19T17:50:09Z)
• MOVE: Effective and Harmless Ownership Verification via Embedded External Features [109.19238806106426]
  We propose an effective and harmless model ownership verification (MOVE) to defend against different types of model stealing simultaneously. We conduct the ownership verification by verifying whether a suspicious model contains the knowledge of defender-specified external features. In particular, we develop our MOVE method under both white-box and black-box settings to provide comprehensive model protection.
  arXiv  Detail & Related papers  (2022-08-04T02:22:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.