BioRefusalAudit: Auditing Biosecurity Refusal Depth Using General and Domain-Fine-Tuned Sparse Autoencoders

• URL: http://arxiv.org/abs/2605.30162v1
• Date: Thu, 28 May 2026 16:18:07 GMT
• Title: BioRefusalAudit: Auditing Biosecurity Refusal Depth Using General and Domain-Fine-Tuned Sparse Autoencoders
• Abstract summary: Across five architectures, no model cleanly discriminated benign from hazard.<n>We tested a panel of Schedule I but biologically non-toxic compounds.<n>Preliminary evidence suggests activation-level auditing may surface failure modes invisible to behavioral evaluation.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Biosecurity evaluations of language models typically ask whether models produce hazardous output. This paper asks a complementary question: when a model refuses, is that refusal structurally sound, or does it disappear under modest changes to prompt framing, formatting, or output length? Across five architectures, no model cleanly discriminated benign from hazard. Gemma 2 2B-IT never genuinely refused across 75 prompts, hedging on every hazard-adjacent query. Gemma 4 E2B-IT refused 65/75 prompts with chat-template formatting and 0/75 without it. Both Gemma models collapsed to 0% under an 80-token cap. Qwen 2.5 1.5B and Phi-3-mini over-refused, flagging 83-87% of benign biology as hazardous. Llama 3.2 1B showed the only meaningful tier gradient (61-point spread). To probe what drives such over-refusal, we tested a panel of Schedule I but biologically non-toxic compounds (notably psilocybin cultivation, with FDA Breakthrough Therapy status). Some models refused these at rates exceeding genuinely hazardous biology, suggesting refusal tracks legality and cultural salience over CBRN hazard. To measure the internal side, we introduce a divergence score D comparing a model's surface response label to its internal sparse autoencoder (SAE) feature activations. Full D was computed on Gemma 2 2B-IT (Gemma Scope 1) and Gemma 4 E2B-IT (author-trained bio SAE). Two fine-tuned Gemma 2 domain SAEs were released. On Gemma 4, comply and refuse responses separated by a 0.647-point gap with zero overlap (n=75), though this is preliminary, with a narrow catalog, within-sample calibration, and Gemma-family-only SAE coverage. Built over one hackathon weekend on consumer hardware (GTX 1650 Ti Max-Q, plus Colab T4 for SAE training), this preliminary evidence suggests activation-level auditing may surface failure modes invisible to behavioral evaluation, with substantial variation across architectures.

Related papers

• Confidently Wrong: Exception Chain Collapse in Frontier LLM Rule Evaluation [0.0]
  We document a failure class in frontier large language models.<n>The failure reproduces at first observation, but its empirical surface is unstable.<n>For regulated, frontier-model accuracy is a moving compliance boundary that shifts without notice.
  arXiv  Detail & Related papers  (2026-07-25T22:40:02Z)
• Abliteration Is Not a Scalpel: Off-Target Effects of Refusal Removal on Decision Disposition Across Model Families [51.56484100374058]
  Abliteration - deleting a model's refusal direction from its weights - is the standard recipe behind popular "uncensored" open-weight models.<n>As a disposition probe we use 21,600 decisions under uncertainty, replayed through a frozen pipeline so the decision-layer model is the only variable.<n>Three effects replicate across both families (weeks-clustered CIs excluding zero)<n>A fourth effect reverses sign: the same operation makes Gemma-abliterated less confident and Qwen-abliterated more (family CIs non-overlapping)
  arXiv  Detail & Related papers  (2026-07-19T22:27:00Z)
• Layer-Isolated Evaluation: Gating the Deterministic Scaffold of a Production LLM Agent with a No-LLM, Regression-Locked Test Harness [45.148328075418156]
  A deployed ordering agent is decomposed into a fixed taxonomy of layers.<n>The pure suite runs in CI on every change against a locked per-slice baseline.<n>We validate by controlled regression injection, degrading one layer at a time across seven non-safety layers.
  arXiv  Detail & Related papers  (2026-06-10T05:55:13Z)
• Cross-Generational Transfer of Adversarial Attacks Reveals Non-Monotonic Safety Alignment in LLMs [0.0]
  We study four generations of Google's Gemma family (7B-31B) with quality-diversity evolution (MAP-Elites) as an automated red-teaming probe.<n>We find that Gemma 3 (12B) exhibits 68.7% +/- 5.7% attack success rate (ASR; mean +/-, 3 seeds), significantly higher than its predecessor Gemma 2.<n> Replaying evolved attack archives across generations reveals that attacks from other generations transfer to Gemma 3 at 44-46% but only 14-18% to Gemma 4.
  arXiv  Detail & Related papers  (2026-05-30T17:07:40Z)
• SomaliBench Eval: Measuring English-to-Somali Refusal Gaps in Open-Weight Language Models [0.0]
  We evaluate four open-weight instruction-tuned models on SomaliBench v0.<n>We find large English-to-Somali refusal gaps for all four models.<n>For three models, the dominant Somali non-refusal mode is not fluent harmful compliance but unclear output.
  arXiv  Detail & Related papers  (2026-05-25T04:45:44Z)
• RefusalBench: Why Refusal Rate Misranks Frontier LLMs on Biological Research Prompts [0.0]
  Strict refusal rates span 0.1% to 94.6% on identical prompts.<n>Nine of 18 frontier models exhibit a hedge-but-help partial-compliance pattern at dual-use tier that binary refusal metrics cannot detect.
  arXiv  Detail & Related papers  (2026-05-20T09:53:31Z)
• Distributional Energy-Based Models for Uncertainty-Aware Structured LLM Reasoning [40.342912574072024]
  Large Language Models produce structured outputs such as travel plans or code solutions.<n>Individual reasoning steps may appear correct while the output as a whole violates budgets, fails test cases, or contradicts earlier deductions.<n>We propose a learned quality scorer with deterministic analytical constraint penalties for verifying structured LLM outputs.
  arXiv  Detail & Related papers  (2026-05-15T17:08:27Z)
• Disposition Distillation at Small Scale: A Three-Arc Negative Result [0.0]
  Internal draft reported +33.9-point MCAS and +15.3-point HumanEval gains on a Qwen3-0.6B student; a second-pass sanity check falsified both numbers before publication.<n>We find no operator that moves judge-measured disposition without damaging content or collapsing into stylistic mimicry.<n>We contribute a three-arc negative result with mechanism, a two-failure-mode taxonomy for linear h_last probes, and an honest falsification pipeline that converts the class of false positives we ourselves produced into publishable negatives.
  arXiv  Detail & Related papers  (2026-04-13T17:40:31Z)
• DyePack: Provably Flagging Test Set Contamination in LLMs Using Backdoors [52.85182605005619]
  We introduce DyePack, a framework that leverages backdoor attacks to identify models that used benchmark test sets during training.<n>Like how banks mix dye packs with their money to mark robbers, DyePack mixes backdoor samples with the test data to flag models that trained on it.<n>We evaluate DyePack on five models across three datasets, covering both multiple-choice and open-ended generation tasks.
  arXiv  Detail & Related papers  (2025-05-29T02:22:14Z)
• Gemma 3 Technical Report [198.3299202423321]
  We introduce Gemma 3, a multimodal addition to the Gemma family of lightweight open models.<n>This version introduces vision understanding abilities, a wider coverage of languages and longer context.<n>We also change the architecture of the model to reduce the KV-cache memory that tends to explode with long context.
  arXiv  Detail & Related papers  (2025-03-25T15:52:34Z)
• Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2 [11.169778211035826]
  Sparse autoencoders (SAEs) are an unsupervised method for learning a neural network's latent representations into seemingly interpretable features. In this work, we introduce Gemma Scope, an open suite of JumpReLU SAEs trained on all layers and sub-layers of Gemma 2 2B and 9B. We evaluate the quality of each SAE on standard metrics and release these results.
  arXiv  Detail & Related papers  (2024-08-09T16:06:42Z)
• Reproducibility in Multiple Instance Learning: A Case For Algorithmic Unit Tests [59.623267208433255]
  Multiple Instance Learning (MIL) is a sub-domain of classification problems with positive and negative labels and a "bag" of inputs. In this work, we examine five of the most prominent deep-MIL models and find that none of them respects the standard MIL assumption. We identify and demonstrate this problem via a proposed "algorithmic unit test", where we create synthetic datasets that can be solved by a MIL respecting model.
  arXiv  Detail & Related papers  (2023-10-27T03:05:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.