Dynamic Evaluation for Oversensitivity in LLMs

• URL: http://arxiv.org/abs/2510.19005v1
• Date: Tue, 21 Oct 2025 18:33:47 GMT
• Title: Dynamic Evaluation for Oversensitivity in LLMs
• Abstract summary: Oversensitivity occurs when language models defensively reject prompts that are actually benign.<n>This behavior not only disrupts user interactions but also obscures the boundary between harmful and harmless content.<n>Existing benchmarks rely on static datasets that degrade overtime as models evolve.
• Score: 68.27609301865174
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Oversensitivity occurs when language models defensively reject prompts that are actually benign. This behavior not only disrupts user interactions but also obscures the boundary between harmful and harmless content. Existing benchmarks rely on static datasets that degrade overtime as models evolve, leading to data contamination and diminished evaluative power. To address this, we develop a framework that dynamically generates model-specific challenging datasets, capturing emerging defensive patterns and aligning with each model's unique behavior. Building on this approach, we construct OVERBENCH, a benchmark that aggregates these datasets across diverse LLM families, encompassing 450,000 samples from 25 models. OVERBENCH provides a dynamic and evolving perspective on oversensitivity, allowing for continuous monitoring of defensive triggers as models advance, highlighting vulnerabilities that static datasets overlook.

Related papers

• Joint Model and Data Sparsification via the Marginal Likelihood [53.29070892356214]
  We propose jointly learning individual feature and sample relevancies, enabling simultaneous model and data sparsification.<n>This symmetric pruning of model and data offers a natural extension that preserves conjugacy.<n> Empirical results across diverse regression tasks affirm that a joint ARD approach consistently yields both sparse and robust prediction models.
  arXiv  Detail & Related papers  (2026-05-28T13:26:53Z)
• Monitoring the Internal Monologue: Probe Trajectories Reveal Reasoning Dynamics [42.27928681606673]
  Large Reasoning Models (LRMs) introduce new opportunities for safety monitoring through their Chain of Thought (CoT) reasoning.<n>We investigate the hidden representations of LRMs to determine whether future behavior can be predicted from prompt and CoT representations.<n>We find that future model behavior is more distinguishable when examined over the full trajectory than from a single static prediction.
  arXiv  Detail & Related papers  (2026-05-18T15:29:04Z)
• Causal Parametric Drift Simulation: A Digital Twin Framework for Classifier Robustness Evaluation [0.0]
  Machine learning classifiers in dynamic environments face concept drift.<n>We propose a framework that complements existing drift detection by leveraging Structural Causal Models as "Digital Twins"<n>Our technique, Causal Parametric Drift Simulation, stress-tests classifiers to identify vulnerabilities before deployment.
  arXiv  Detail & Related papers  (2026-05-10T17:28:49Z)
• Reinforcement-Guided Synthetic Data Generation for Privacy-Sensitive Identity Recognition [60.52810518437911]
  High-fidelity generative models are increasingly needed in privacy-sensitive scenarios.<n> limited data leads to poor generative models, which in turn fail to mitigate data scarcity.<n>We propose a reinforcement-guided synthetic data generation framework that adapts general-domain generative priors to identity recognition tasks.
  arXiv  Detail & Related papers  (2026-04-09T06:52:03Z)
• Brittlebench: Quantifying LLM robustness via prompt sensitivity [44.950999933205985]
  We introduce a theoretical framework for quantifying model sensitivity to prompt variants, or brittleness.<n>We design a novel evaluation pipeline, Brittlebench, to holistically evaluate the sensitivity of frontier models.<n>We find that semantics-preserving input perturbations can account for up to half of the performance variance for a given model.
  arXiv  Detail & Related papers  (2026-02-27T21:12:13Z)
• From Lab to Reality: A Practical Evaluation of Deep Learning Models and LLMs for Vulnerability Detection [2.8647133890967]
  Vulnerability detection methods based on deep learning (DL) have shown strong performance on benchmark datasets, yet their real-world effectiveness remains underexplored.<n>Recent work suggests that both graph neural network (GNN)-based and transformer-based models, including large language models (LLMs) yield promising results when evaluated on curated benchmark datasets.<n>In this study, we systematically evaluate two representative DL models-ReVeal and LineVul-across four representative datasets.
  arXiv  Detail & Related papers  (2025-12-11T10:04:54Z)
• Token-Level Marginalization for Multi-Label LLM Classifiers [0.0]
  Three novel token-level probability estimation approaches are proposed.<n>The aim is to enhance model interpretability and accuracy, and evaluate the generalizability of this framework across different instruction-tuned models.
  arXiv  Detail & Related papers  (2025-11-27T10:43:26Z)
• RoHOI: Robustness Benchmark for Human-Object Interaction Detection [84.78366452133514]
  Human-Object Interaction (HOI) detection is crucial for robot-human assistance, enabling context-aware support.<n>We introduce the first benchmark for HOI detection, evaluating model resilience under diverse challenges.<n>Our benchmark, RoHOI, includes 20 corruption types based on the HICO-DET and V-COCO datasets and a new robustness-focused metric.
  arXiv  Detail & Related papers  (2025-07-12T01:58:04Z)
• Accidental Vulnerability: Factors in Fine-Tuning that Shift Model Safeguards [13.197807179926428]
  Large language models (LLMs) gain popularity, their vulnerability to adversarial attacks emerges as a primary concern.<n>In this work, we investigate Accidental Vulnerability, unexpected vulnerabilities arising from characteristics of fine-tuning data.
  arXiv  Detail & Related papers  (2025-05-22T15:30:00Z)
• Byzantine-Robust Federated Learning Using Generative Adversarial Networks [1.4091801425319963]
  Federated learning (FL) enables collaborative model training across distributed clients without sharing raw data, but its robustness is threatened by Byzantine behaviors such as data and model poisoning.<n>We present a defense framework that addresses these challenges by leveraging a conditional generative adversarial network (cGAN) at the server to synthesize representative data for validating client updates.<n>This approach eliminates reliance on external datasets, adapts to diverse attack strategies, and integrates seamlessly into standard FL.
  arXiv  Detail & Related papers  (2025-03-26T18:00:56Z)
• Towards Evaluating the Robustness of Visual State Space Models [63.14954591606638]
  Vision State Space Models (VSSMs) have demonstrated remarkable performance in visual perception tasks. However, their robustness under natural and adversarial perturbations remains a critical concern. We present a comprehensive evaluation of VSSMs' robustness under various perturbation scenarios.
  arXiv  Detail & Related papers  (2024-06-13T17:59:44Z)
• ReEval: Automatic Hallucination Evaluation for Retrieval-Augmented Large Language Models via Transferable Adversarial Attacks [91.55895047448249]
  This paper presents ReEval, an LLM-based framework using prompt chaining to perturb the original evidence for generating new test cases. We implement ReEval using ChatGPT and evaluate the resulting variants of two popular open-domain QA datasets. Our generated data is human-readable and useful to trigger hallucination in large language models.
  arXiv  Detail & Related papers  (2023-10-19T06:37:32Z)
• Bring Your Own Data! Self-Supervised Evaluation for Large Language Models [52.15056231665816]
  We propose a framework for self-supervised evaluation of Large Language Models (LLMs) We demonstrate self-supervised evaluation strategies for measuring closed-book knowledge, toxicity, and long-range context dependence. We find strong correlations between self-supervised and human-supervised evaluations.
  arXiv  Detail & Related papers  (2023-06-23T17:59:09Z)
• CausalAgents: A Robustness Benchmark for Motion Forecasting using Causal Relationships [8.679073301435265]
  We construct a new benchmark for evaluating and improving model robustness by applying perturbations to existing data. We use these labels to perturb the data by deleting non-causal agents from the scene. Under non-causal perturbations, we observe a $25$-$38%$ relative change in minADE as compared to the original.
  arXiv  Detail & Related papers  (2022-07-07T21:28:23Z)
• Generative Modeling Helps Weak Supervision (and Vice Versa) [87.62271390571837]
  We propose a model fusing weak supervision and generative adversarial networks. It captures discrete variables in the data alongside the weak supervision derived label estimate. It is the first approach to enable data augmentation through weakly supervised synthetic images and pseudolabels.
  arXiv  Detail & Related papers  (2022-03-22T20:24:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.