Surrogate Fidelity: When Can Open LLMs Explain Closed Ones?

• URL: http://arxiv.org/abs/2606.32008v1
• Date: Tue, 30 Jun 2026 17:43:44 GMT
• Title: Surrogate Fidelity: When Can Open LLMs Explain Closed Ones?
• Abstract summary: We evaluate surrogate fidelity at the prediction, attribution, and representation levels.<n>We document an access-validity inversion: white-box signals like attention patterns and magnitudes are highly stable across models but only weakly predictive of causal attributions.
• Score: 6.786301369955054
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Mechanistic interpretability (MI) requires full access to model internals, yet the APIs for most widely deployed language models at best expose log-probabilities over output tokens. This creates a surrogate problem: when do measurements made on open models allow us to make claims about a closed model? We evaluate surrogate fidelity at the prediction, attribution, and representation levels. For binary classification tasks, log-odds provide an API-compatible scalar readout of the model's representation space, and leave-one-out attributions provide insight into model behavior. Across eleven models spanning four families (Llama, Qwen, GPT, and Gemini), we find that prediction fidelity substantially overstates attribution fidelity: models that agree on what the answer is often disagree on why. We document an access-validity inversion: white-box signals like attention patterns and perturbation magnitudes are highly stable across models but only weakly predictive of causal attributions, which black-box input ablations capture by design. Mechanistic insight does not automatically transfer to closed targets, and prediction-level agreement is insufficient to warrant such transfer. Code and results are available at https://github.com/facebookresearch/surrogate.

Related papers

• Catch-Only-One: Non-Transferable Examples for Model-Specific Authorization [26.668781698446832]
  Recent AI regulations call for data that remain useful for innovation while resistant to misuse.<n>We propose non-transferable examples (NEs), a training-free and data-agnostic input-side usage-control mechanism.<n>We establish formal bounds that guarantee utility for the authorized model and quantify deviation for unauthorized ones.
  arXiv  Detail & Related papers  (2025-10-13T03:43:11Z)
• Watch the Weights: Unsupervised monitoring and control of fine-tuned LLMs [27.544312683007234]
  We introduce a new method for understanding, monitoring and controlling fine-tuned large language models (LLMs)<n>We demonstrate that the top singular of the weight difference between a fine-tuned model and its base model correspond to newly acquired behaviors.<n>For backdoored models that bypasses safety mechanisms when a secret trigger is present, our method stops up to 100% of attacks with a false positive rate below 1.2%.
  arXiv  Detail & Related papers  (2025-07-31T21:04:12Z)
• Predicting the Performance of Black-box LLMs through Self-Queries [60.87193950962585]
  Large language models (LLMs) are increasingly relied on in AI systems, predicting when they make mistakes is crucial.<n>In this paper, we extract features of LLMs in a black-box manner by using follow-up prompts and taking the probabilities of different responses as representations.<n>We demonstrate that training a linear model on these low-dimensional representations produces reliable predictors of model performance at the instance level.
  arXiv  Detail & Related papers  (2025-01-02T22:26:54Z)
• Promises and Pitfalls of Generative Masked Language Modeling: Theoretical Framework and Practical Guidelines [74.42485647685272]
  We focus on Generative Masked Language Models (GMLMs) We train a model to fit conditional probabilities of the data distribution via masking, which are subsequently used as inputs to a Markov Chain to draw samples from the model. We adapt the T5 model for iteratively-refined parallel decoding, achieving 2-3x speedup in machine translation with minimal sacrifice in quality.
  arXiv  Detail & Related papers  (2024-07-22T18:00:00Z)
• Co(ve)rtex: ML Models as storage channels and their (mis-)applications [2.792027541710663]
  In machine learning systems, don't-care states and undefined behavior have been shown to be sources of significant vulnerabilities. We consider the ML model as a storage channel with a capacity that increases with over parameterization. We develop optimizations to improve the capacity in this case, including a novel ML-specific substitution based error correction protocol.
  arXiv  Detail & Related papers  (2023-07-17T19:57:10Z)
• Black-Box Anomaly Attribution [13.455748795087493]
  When a black-box machine learning model deviates from the true observation, what can be said about the reason behind that deviation? This is a fundamental and ubiquitous question that the end user in a business or industrial AI application often asks. We propose a novel likelihood-based attribution framework we call the likelihood compensation''
  arXiv  Detail & Related papers  (2023-05-29T01:42:32Z)
• VisFIS: Visual Feature Importance Supervision with Right-for-the-Right-Reason Objectives [84.48039784446166]
  We show that model FI supervision can meaningfully improve VQA model accuracy as well as performance on several Right-for-the-Right-Reason metrics. Our best performing method, Visual Feature Importance Supervision (VisFIS), outperforms strong baselines on benchmark VQA datasets. Predictions are more accurate when explanations are plausible and faithful, and not when they are plausible but not faithful.
  arXiv  Detail & Related papers  (2022-06-22T17:02:01Z)
• Probing Model Signal-Awareness via Prediction-Preserving Input Minimization [67.62847721118142]
  We evaluate models' ability to capture the correct vulnerability signals to produce their predictions. We measure the signal awareness of models using a new metric we propose- Signal-aware Recall (SAR) The results show a sharp drop in the model's Recall from the high 90s to sub-60s with the new metric.
  arXiv  Detail & Related papers  (2020-11-25T20:05:23Z)
• How do Decisions Emerge across Layers in Neural Models? Interpretation with Differentiable Masking [70.92463223410225]
  DiffMask learns to mask-out subsets of the input while maintaining differentiability. Decision to include or disregard an input token is made with a simple model based on intermediate hidden layers. This lets us not only plot attribution heatmaps but also analyze how decisions are formed across network layers.
  arXiv  Detail & Related papers  (2020-04-30T17:36:14Z)
• AvgOut: A Simple Output-Probability Measure to Eliminate Dull Responses [97.50616524350123]
  We build dialogue models that are dynamically aware of what utterances or tokens are dull without any feature-engineering. The first model, MinAvgOut, directly maximizes the diversity score through the output distributions of each batch. The second model, Label Fine-Tuning (LFT), prepends to the source sequence a label continuously scaled by the diversity score to control the diversity level. The third model, RL, adopts Reinforcement Learning and treats the diversity score as a reward signal.
  arXiv  Detail & Related papers  (2020-01-15T18:32:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.