Evaluation without Generation: Non-Generative Assessment of Harmful Model Specialization with Applications to CSAM

• URL: http://arxiv.org/abs/2604.25119v1
• Date: Tue, 28 Apr 2026 01:54:25 GMT
• Title: Evaluation without Generation: Non-Generative Assessment of Harmful Model Specialization with Applications to CSAM
• Abstract summary: We introduce Gaussian probing, a method that characterizes how LoRA adaptors perturb a model's internal representations by measuring responses to latent ensembles.<n>We demonstrate effectiveness in high-risk domains, including detecting models specialized for child sexual abuse material.<n>Our results show that Gaussian probing provides a scalable non-generative alternative for evaluating high-risk generative systems.
• Score: 24.60847002212009
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Auditing the fine-tunes of open-weight generative models for harmful specialization has become a new governance challenge for model hosting platforms. The standard toolkit, generative evaluation via curated prompts or red-teaming, does not scale to platform-level auditing and breaks down entirely for domains like CSAM where generation is legally constrained. This motivates the Evaluation without Generation problem: assessing model capabilities without producing outputs. We argue that in such settings, capability must be inferred from the model's state, either its parameters or internal representations, rather than its outputs. We introduce Gaussian probing, a method that characterizes how LoRA adaptors perturb a model's internal representations by measuring responses to Gaussian latent ensembles. Unlike raw-weight baselines, Gaussian probing reliably distinguishes benign from harmful specialization without sampling outputs. We demonstrate effectiveness in high-risk domains, including detecting models specialized for child sexual abuse material (CSAM), where output-based evaluation is legally and ethically constrained. Our results show that Gaussian probing provides a scalable non-generative alternative for evaluating high-risk generative systems and remains robust to weight rescaling, a representative adversarial manipulation.

Related papers

• Automatic, Debiased, and Invariant Counterfactual Generation under General Interventions [7.1743353314344835]
  ADIGen is a framework for automatic, debiased, and invariant counterfactual generation under general interventions.<n>We show that ADIGen controls counterfactual risk under general interventions, with a product-bias nuisance and an invariant risk bound across environments.
  arXiv  Detail & Related papers  (2026-06-05T15:40:59Z)
• Revisiting Vul-RAG: Reproducibility and Replicability of RAG-based Vulnerability Detection with Open-Weight Models [0.2730969268472861]
  We present a study of Vul-RAG, a framework for source code vulnerability detection that generalizes with high-level vulnerability knowledge.<n>Results confirm that the findings of Vul-RAG are reproducible under local deployment, but with minor deviations.<n>We discuss practical implications and trade-offs between detection effectiveness, model capabilities, and model scale.
  arXiv  Detail & Related papers  (2026-06-03T11:20:37Z)
• Non-Parametric Rehearsal Learning via Conditional Mean Embeddings [88.89267783967263]
  We propose a non-parametric rehearsal learning approach for textitavoiding undesired future (AUF) problems.<n>Specifically, we use kernel machinery to reformulate the AUF objective into a unified representation that disentangles desirability modeling from action-induced distributional changes.
  arXiv  Detail & Related papers  (2026-05-09T15:30:52Z)
• LiveFMBench: Unveiling the Power and Limits of Agentic Workflows in Specification Generation [75.05397479715576]
  Large language models (LLMs) and agents have shown promising progress, but their true capabilities and failure modes remain unclear.<n>We present the first systematic and contamination-aware study of LLM- and agent-based formal specification generation for C programs.
  arXiv  Detail & Related papers  (2026-05-02T11:31:33Z)
• Sampler-Robust Optimization under Generative Models [4.250238840479483]
  Decision pipelines increasingly rely on learned generative models to represent uncertainty.<n> Reliability depends on two errors: sampler misspecification and finite-simulation error.<n>We propose sampler-Robust Optimization, which optimize decisions against the worst-case sampler induced by perturbing the learned generator.
  arXiv  Detail & Related papers  (2026-04-30T05:33:15Z)
• Conformal Selective Prediction with General Risk Control [8.936337121607677]
  We propose a new framework for deriving such decisions for any trained model and any user-defined, bounded and continuously-valued risk.<n>SCoRE offers two types of guarantees on the risk among positive'' cases in which the system opts to trust the model.<n>We evaluate the proposed methods with simulations and demonstrate their efficacy through applications to error management in drug discovery, health risk prediction, and large language models.
  arXiv  Detail & Related papers  (2026-03-25T18:29:23Z)
• Efficient Adaptive Rejection Sampling for Accelerating Speculative Decoding in Large Language Models [2.4065240342323384]
  This paper introduces Efficient Adaptive Rejection Sampling (EARS)<n>EARS dynamically adjusts the acceptance threshold by incorporating the target model's own predictive uncertainty, measured as 1 - max(P_target)<n>It significantly enhances the efficiency of speculative decoding, achieving up to an 18.12% increase in throughput with a negligible 0.84% accuracy drop on the GSM8K benchmark.
  arXiv  Detail & Related papers  (2025-12-15T11:08:56Z)
• Principled Input-Output-Conditioned Post-Hoc Uncertainty Estimation for Regression Networks [1.4671424999873808]
  Uncertainty is critical in safety-sensitive applications but is often omitted from off-the-shelf neural networks due to adverse effects on predictive performance.<n>We propose a theoretically grounded framework for post-hoc uncertainty estimation in regression tasks by fitting an auxiliary model to both original inputs and frozen model outputs.
  arXiv  Detail & Related papers  (2025-06-01T09:13:27Z)
• Uncertainty-Aware Decoding with Minimum Bayes Risk [70.6645260214115]
  We show how Minimum Bayes Risk decoding, which selects model generations according to an expected risk, can be generalized into a principled uncertainty-aware decoding method.<n>We show that this modified expected risk is useful for both choosing outputs and deciding when to abstain from generation and can provide improvements without incurring overhead.
  arXiv  Detail & Related papers  (2025-03-07T10:55:12Z)
• Controlling Risk of Retrieval-augmented Generation: A Counterfactual Prompting Framework [77.45983464131977]
  We focus on how likely it is that a RAG model's prediction is incorrect, resulting in uncontrollable risks in real-world applications.<n>Our research identifies two critical latent factors affecting RAG's confidence in its predictions.<n>We develop a counterfactual prompting framework that induces the models to alter these factors and analyzes the effect on their answers.
  arXiv  Detail & Related papers  (2024-09-24T14:52:14Z)
• Risk-Sensitive Diffusion: Robustly Optimizing Diffusion Models with Noisy Samples [58.68233326265417]
  Non-image data are prevalent in real applications and tend to be noisy. Risk-sensitive SDE is a type of differential equation (SDE) parameterized by the risk vector. We conduct systematic studies for both Gaussian and non-Gaussian noise distributions.
  arXiv  Detail & Related papers  (2024-02-03T08:41:51Z)
• Toward Certified Robustness Against Real-World Distribution Shifts [65.66374339500025]
  We train a generative model to learn perturbations from data and define specifications with respect to the output of the learned model. A unique challenge arising from this setting is that existing verifiers cannot tightly approximate sigmoid activations. We propose a general meta-algorithm for handling sigmoid activations which leverages classical notions of counter-example-guided abstraction refinement.
  arXiv  Detail & Related papers  (2022-06-08T04:09:13Z)
• OMASGAN: Out-of-Distribution Minimum Anomaly Score GAN for Sample Generation on the Boundary [0.0]
  Generative models set high likelihood and low reconstruction loss to Out-of-Distribution (OoD) samples. OMASGAN generates, in a negative data augmentation manner, anomalous samples on the estimated distribution boundary. OMASGAN performs retraining by including the abnormal minimum-anomaly-score OoD samples generated on the distribution boundary.
  arXiv  Detail & Related papers  (2021-10-28T16:35:30Z)
• Continual Learning with Fully Probabilistic Models [70.3497683558609]
  We present an approach for continual learning based on fully probabilistic (or generative) models of machine learning. We propose a pseudo-rehearsal approach using a Gaussian Mixture Model (GMM) instance for both generator and classifier functionalities. We show that GMR achieves state-of-the-art performance on common class-incremental learning problems at very competitive time and memory complexity.
  arXiv  Detail & Related papers  (2021-04-19T12:26:26Z)
• Generalized Multi-Output Gaussian Process Censored Regression [7.111443975103331]
  We introduce a heteroscedastic multi-output Gaussian process model which combines the non-parametric flexibility of GPs with the ability to leverage information from correlated outputs under input-dependent noise conditions. Results show how the added flexibility allows our model to better estimate the underlying non-censored (i.e. true) process under potentially complex censoring dynamics.
  arXiv  Detail & Related papers  (2020-09-10T12:46:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.