Bias Ahead: Sensitive Prompts as Early Warnings for Fairness in Large Language Models

• URL: http://arxiv.org/abs/2604.05575v1
• Date: Tue, 07 Apr 2026 08:16:02 GMT
• Title: Bias Ahead: Sensitive Prompts as Early Warnings for Fairness in Large Language Models
• Abstract summary: We propose sensitive prompts as a new abstraction for fairness evaluation.<n>We construct and release SensY, a dataset of 12,801 prompts, categorized as sensitive and non-sensitive.<n>We analyze 4,500 responses to evaluate their adequacy in answering sensitive prompts.
• Score: 44.425146021311434
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large Language Models (LLMs) are being increasingly integrated into software systems, offering powerful capabilities but also raising concerns about fairness. Existing fairness benchmarks, however, focus on stereotype-specific associations, which limit their ability to anticipate risks in diverse, real-world contexts. In this paper, we propose sensitive prompts as a new abstraction for fairness evaluation: inputs that are not inherently biased but are more likely to elicit biased or inadequate responses due to the sensitivity of their content. We construct and release SensY, a dataset of 12,801 prompts, categorized as sensitive and non-sensitive, spanning seven thematic domains, combining synthetic generation and real user inputs. Using this dataset, we query three open-source LLMs and manually analyze 4,500 responses to evaluate their adequacy in answering sensitive prompts. Results show that while models often provide factually correct answers, they frequently fail to acknowledge the ethical, relational, or contextual implications of sensitive inputs. In addition, we develop an automated classifier for predicting prompt sensitivity, achieving robust performance on sensitive prompts. Our findings demonstrate that prompt sensitivity can serve as an effective early-warning mechanism for fairness risks in LLMs. This perspective shifts fairness assessment from reactive mitigation toward preventive design, enabling developers to anticipate and manage bias before deployment.

Related papers

• PromptAudit: Auditing Prompt Sensitivity in LLM-Based Vulnerability Detection [11.513151833477837]
  We evaluate accuracy, recall, abstention, coverage, and effective F1 using five prompting strategies across five open-weight models on 1,000 CVEs.<n>We find that standard chain-of-thought prompting achieves the strongest overall operational performance.<n> adaptive chain-of-thought frequently suppresses recall and self-consistency induces excessive abstention.
  arXiv  Detail & Related papers  (2026-05-22T19:44:51Z)
• Improving Recommendation Fairness without Sensitive Attributes Using Multi-Persona LLMs [21.381646091763272]
  We aim to improve recommendation fairness without access to sensitive attributes.<n>We propose a novel framework for Fair recommendation withOut Sensitive Attributes (LLMFOSA)
  arXiv  Detail & Related papers  (2025-05-26T03:52:41Z)
• Gensors: Authoring Personalized Visual Sensors with Multimodal Foundation Models and Reasoning [61.17099595835263]
  Gensors is a system that empowers users to define customized sensors supported by the reasoning capabilities of MLLMs.<n>In a user study, participants reported significantly greater sense of control, understanding, and ease of communication when defining sensors using Gensors.
  arXiv  Detail & Related papers  (2025-01-27T01:47:57Z)
• On the Fairness, Diversity and Reliability of Text-to-Image Generative Models [68.62012304574012]
  multimodal generative models have sparked critical discussions on their reliability, fairness and potential for misuse.<n>We propose an evaluation framework to assess model reliability by analyzing responses to global and local perturbations in the embedding space.<n>Our method lays the groundwork for detecting unreliable, bias-injected models and tracing the provenance of embedded biases.
  arXiv  Detail & Related papers  (2024-11-21T09:46:55Z)
• ProSA: Assessing and Understanding the Prompt Sensitivity of LLMs [72.13489820420726]
  ProSA is a framework designed to evaluate and comprehend prompt sensitivity in large language models. Our study uncovers that prompt sensitivity fluctuates across datasets and models, with larger models exhibiting enhanced robustness.
  arXiv  Detail & Related papers  (2024-10-16T09:38:13Z)
• How are Prompts Different in Terms of Sensitivity? [50.67313477651395]
  We present a comprehensive prompt analysis based on the sensitivity of a function. We use gradient-based saliency scores to empirically demonstrate how different prompts affect the relevance of input tokens to the output. We introduce sensitivity-aware decoding which incorporates sensitivity estimation as a penalty term in the standard greedy decoding.
  arXiv  Detail & Related papers  (2023-11-13T10:52:01Z)
• Balancing Robustness and Sensitivity using Feature Contrastive Learning [95.86909855412601]
  Methods that promote robustness can hurt the model's sensitivity to rare or underrepresented patterns. We propose Feature Contrastive Learning (FCL) that encourages a model to be more sensitive to the features that have higher contextual utility.
  arXiv  Detail & Related papers  (2021-05-19T20:53:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.