Position: Explaining Behavioral Shifts in Large Language Models Requires a Comparative Approach

• URL: http://arxiv.org/abs/2602.02304v1
• Date: Mon, 02 Feb 2026 16:36:21 GMT
• Title: Position: Explaining Behavioral Shifts in Large Language Models Requires a Comparative Approach
• Abstract summary: We argue that behavioral shifts should be explained comparatively, rather than any single model in isolation.<n>We formulate a Comparative XAI ($$-XAI) framework with a set of desiderata to be taken into account when designing proper explaining methods.
• Score: 7.72875981545154
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large-scale foundation models exhibit behavioral shifts: intervention-induced behavioral changes that appear after scaling, fine-tuning, reinforcement learning or in-context learning. While investigating these phenomena have recently received attention, explaining their appearance is still overlooked. Classic explainable AI (XAI) methods can surface failures at a single checkpoint of a model, but they are structurally ill-suited to justify what changed internally across different checkpoints and which explanatory claims are warranted about that change. We take the position that behavioral shifts should be explained comparatively: the core target should be the intervention-induced shift between a reference model and an intervened model, rather than any single model in isolation. To this aim we formulate a Comparative XAI ($Δ$-XAI) framework with a set of desiderata to be taken into account when designing proper explaining methods. To highlight how $Δ$-XAI methods work, we introduce a set of possible pipelines, relate them to the desiderata, and provide a concrete $Δ$-XAI experiment.

Related papers

• Probing Perceptual Priors of MLLMs via Gibbs Sampling with Interpretable Generative Controls [30.071130311851277]
  A model's behavior on a task is jointly determined by the input it receives and the prior it brings in, i.e. the distribution over stimuli it implicitly expects.<n>We propose a method to sample from models' perceptual prior distributions directly, by steering a generative model to produce stimuli along controllable axes and running Gibbs sampling over that space with the model under study as the judge.
  arXiv  Detail & Related papers  (2026-08-27T21:38:41Z)
• Dissecting model behavior through agent trajectories [15.811597127775812]
  We formalize the intent-execution' gap: the mismatch between what the model intends and what the harness executes, and vice versa.<n>To illustrate the impact of this harness-model alignment, we develop a simple and customizable harness called Simple Strands Agent' (SSA)<n>We make two contributions: (i) we reproduce or improve on the pass@1 performance reported by diverse model-provider families on popular agentic benchmarks, and (ii) building on an analysis of 138k trajectories generated by SSA.
  arXiv  Detail & Related papers  (2026-06-16T03:17:03Z)
• Automatically Finding and Validating Unexpected Side-Effects of Interventions on Language Models [0.9099663022952497]
  We present an automated, contrastive evaluation pipeline for auditing the behavioral impact of interventions on large language models.<n>We evaluate the approach in synthetic setting by injecting known behavioral changes and showing that the pipeline reliably recovers them.<n>Overall, the pipeline provides a statistically grounded and interpretable tool for post-hoc auditing of intervention-induced changes in model behavior.
  arXiv  Detail & Related papers  (2026-05-06T16:27:23Z)
• Neural Network Reprogrammability: A Unified Theme on Model Reprogramming, Prompt Tuning, and Prompt Instruction [57.19302613163439]
  We introduce neural network reprogrammability as a unifying framework for model adaptation.<n>We present a taxonomy that categorizes such information manipulation approaches across four key dimensions.<n>We also analyze remaining technical challenges and ethical considerations.
  arXiv  Detail & Related papers  (2025-06-05T05:42:27Z)
• ExPLAIND: Unifying Model, Data, and Training Attribution to Study Model Behavior [39.590138981646696]
  Post-hoc interpretability methods typically attribute a model's behavior to its components, data, or training trajectory in isolation.<n>We present ExPLAIND, a unified framework that integrates all these perspectives.
  arXiv  Detail & Related papers  (2025-05-26T14:53:11Z)
• Internal Causal Mechanisms Robustly Predict Language Model Out-of-Distribution Behaviors [61.92704516732144]
  We show that the most robust features for correctness prediction are those that play a distinctive causal role in the model's behavior.<n>We propose two methods that leverage causal mechanisms to predict the correctness of model outputs.
  arXiv  Detail & Related papers  (2025-05-17T00:31:39Z)
• Overcoming Sparsity Artifacts in Crosscoders to Interpret Chat-Tuning [9.761667621068787]
  Crosscoders are a recent model diffing method that learns a shared dictionary of interpretable concepts represented as latent directions in both the base and fine-tuned models.<n>We identify two issues which stem from the crosscoders L1 training loss that can misattribute concepts as unique to the fine-tuned model, when they really exist in both models.<n>We train a crosscoder with BatchTopK loss and show that it substantially mitigates these issues, finding more genuinely chat-specific and highly interpretable concepts.
  arXiv  Detail & Related papers  (2025-04-03T17:50:24Z)
• Explain, Edit, and Understand: Rethinking User Study Design for Evaluating Model Explanations [97.91630330328815]
  We conduct a crowdsourcing study, where participants interact with deception detection models that have been trained to distinguish between genuine and fake hotel reviews. We observe that for a linear bag-of-words model, participants with access to the feature coefficients during training are able to cause a larger reduction in model confidence in the testing phase when compared to the no-explanation control.
  arXiv  Detail & Related papers  (2021-12-17T18:29:56Z)
• Influence Tuning: Demoting Spurious Correlations via Instance Attribution and Instance-Driven Updates [26.527311287924995]
  influence tuning can help deconfounding the model from spurious patterns in data. We show that in a controlled setup, influence tuning can help deconfounding the model from spurious patterns in data.
  arXiv  Detail & Related papers  (2021-10-07T06:59:46Z)
• Beyond Trivial Counterfactual Explanations with Diverse Valuable Explanations [64.85696493596821]
  In computer vision applications, generative counterfactual methods indicate how to perturb a model's input to change its prediction. We propose a counterfactual method that learns a perturbation in a disentangled latent space that is constrained using a diversity-enforcing loss. Our model improves the success rate of producing high-quality valuable explanations when compared to previous state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-03-18T12:57:34Z)
• Generative Temporal Difference Learning for Infinite-Horizon Prediction [101.59882753763888]
  We introduce the $gamma$-model, a predictive model of environment dynamics with an infinite probabilistic horizon. We discuss how its training reflects an inescapable tradeoff between training-time and testing-time compounding errors.
  arXiv  Detail & Related papers  (2020-10-27T17:54:12Z)
• Deducing neighborhoods of classes from a fitted model [68.8204255655161]
  In this article a new kind of interpretable machine learning method is presented. It can help to understand the partitioning of the feature space into predicted classes in a classification model using quantile shifts. Basically, real data points (or specific points of interest) are used and the changes of the prediction after slightly raising or decreasing specific features are observed.
  arXiv  Detail & Related papers  (2020-09-11T16:35:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.