Mitigating Prompt-Induced Cognitive Biases in General-Purpose AI for Software Engineering

• URL: http://arxiv.org/abs/2604.16756v2
• Date: Wed, 22 Apr 2026 10:51:32 GMT
• Title: Mitigating Prompt-Induced Cognitive Biases in General-Purpose AI for Software Engineering
• Abstract summary: In software engineering (SE) decision support (where problem statements and requirements are natural language) small phrasing shifts can push GPAI models toward suboptimal decisions.<n>We study this with PROBE-SWE, a benchmark for SE that pairs biased and unbiased versions of the same SE dilemmas.<n>We find no statistically significant reductions in bias sensitivity on a per-bias basis.
• Score: 7.619404259039282
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Prompt-induced cognitive biases are changes in a general-purpose AI (GPAI) system's decisions caused solely by biased wording in the input (e.g., framing, anchors), not task logic. In software engineering (SE) decision support (where problem statements and requirements are natural language) small phrasing shifts (e.g., popularity hints or outcome reveals) can push GPAI models toward suboptimal decisions. We study this with PROBE-SWE, a dynamic benchmark for SE that pairs biased and unbiased versions of the same SE dilemmas, controls for logic and difficulty, and targets eight SE-relevant biases (anchoring, availability, bandwagon, confirmation, framing, hindsight, hyperbolic discounting, overconfidence). We ask whether prompt engineering mitigates bias sensitivity in practice, focusing on actionable techniques that practitioners can apply off-the-shelf in real environments. Testing common strategies (e.g., chain-of-thought, self-debiasing) on cost-effective GPAI systems, we find no statistically significant reductions in bias sensitivity on a per-bias basis. We then adopt a Prolog-style view of the reasoning process: solving SE dilemmas requires making explicit any background axioms and inference assumptions (i.e., SE best practices) that are usually implicit in the prompt. So, we hypothesize that bias-inducing features short-circuit assumption elicitation, pushing GPAI models toward biased shortcuts. Building on this, we introduce an end-to-end method that elicits best practices and injects axiomatic reasoning cues into the prompt before answering, reducing overall bias sensitivity by 51% on average (p < .001). Finally, we report a thematic analysis that surfaces linguistic patterns associated with heightened bias sensitivity, clarifying when GPAI use is less advisable for SE decision support and where to focus future countermeasures.

Related papers

• Selective Test-Time Debiasing for CLIP via Reward Gating [9.837249813808603]
  Vision language models (VLMs) demonstrate strong zero-shot performance, but often perpetuate social stereotypes in person-centric queries.<n>We introduce Reward-Gated Test-Time Adaptation (RG-TTA), a reinforcement learning-based test-time adaptation framework.<n>RG-TTA adaptively triggers fairness regularization based on the bias sensitivity of each input during test-time policy adaptation.
  arXiv  Detail & Related papers  (2026-07-01T04:33:26Z)
• Making Bias Non-Predictive: Training Robust LLM Judges via Reinforcement Learning [91.8584139564909]
  Large language models (LLMs) increasingly serve as automated judges, yet they remain susceptible to cognitive biases.<n>We propose Epistemic Independence Training (EIT), a reinforcement learning framework grounded in a key principle.<n>EIT operationalizes this through a balanced conflict strategy where bias signals are equally likely to support correct and incorrect answers.
  arXiv  Detail & Related papers  (2026-02-02T01:43:48Z)
• Is General-Purpose AI Reasoning Sensitive to Data-Induced Cognitive Biases? Dynamic Benchmarking on Typical Software Engineering Dilemmas [47.582118202259394]
  General-purpose AI (GPAI) systems may help mitigate human cognitive biases due to their non-human nature.<n>But their training on human-generated data raises a critical question: Do GPAI systems themselves exhibit cognitive biases?<n>We present the first benchmarking framework to evaluate data-induced cognitive biases in GPAI within software engineering.
  arXiv  Detail & Related papers  (2025-08-15T07:29:46Z)
• Self-Adaptive Cognitive Debiasing for Large Language Models in Decision-Making [71.71796367760112]
  Large language models (LLMs) have shown potential in supporting decision-making applications.<n>We propose a cognitive debiasing approach, self-adaptive cognitive debiasing (SACD)<n>We evaluate SACD on finance, healthcare, and legal decision-making tasks using both open-weight and closed-weight LLMs.
  arXiv  Detail & Related papers  (2025-04-05T11:23:05Z)
• Does Reasoning Introduce Bias? A Study of Social Bias Evaluation and Mitigation in LLM Reasoning [21.921684911834447]
  We present the first systematic evaluation of social bias within large language models (LLMs)<n>We analyze both prediction accuracy and reasoning bias across a broad spectrum of models, including instruction-tuned and CoT-augmented variants of DeepSeek-R1 and ChatGPT.<n>We propose Answer Distribution as Bias Proxy (ADBP), a lightweight mitigation method that detects bias by tracking how model predictions change across incremental reasoning steps.
  arXiv  Detail & Related papers  (2025-02-21T10:16:07Z)
• ALBAR: Adversarial Learning approach to mitigate Biases in Action Recognition [52.537021302246664]
  Action recognition models often suffer from background bias (i.e., inferring actions based on background cues) and foreground bias (i.e., relying on subject appearance)<n>We propose ALBAR, a novel adversarial training method that mitigates foreground and background biases without requiring specialized knowledge of the bias attributes.<n>We evaluate our method on established background and foreground bias protocols, setting a new state-of-the-art and strongly improving combined debiasing performance by over 12% absolute on HMDB51.
  arXiv  Detail & Related papers  (2025-01-31T20:47:06Z)
• Chain of Thought Still Thinks Fast: APriCoT Helps with Thinking Slow [0.0]
  We introduce Counterfactual Prompting with Agnostically Primed CoT (APriCoT)<n>APriCoT effectively reduces the influence of base-rate probabilities while improving overall accuracy.<n>Our results suggest that mitigating bias requires a slow thinking process which CoT alone may not provide.
  arXiv  Detail & Related papers  (2024-08-16T10:34:50Z)
• Take Care of Your Prompt Bias! Investigating and Mitigating Prompt Bias in Factual Knowledge Extraction [56.17020601803071]
  Recent research shows that pre-trained language models (PLMs) suffer from "prompt bias" in factual knowledge extraction. This paper aims to improve the reliability of existing benchmarks by thoroughly investigating and mitigating prompt bias.
  arXiv  Detail & Related papers  (2024-03-15T02:04:35Z)
• Delving into Identify-Emphasize Paradigm for Combating Unknown Bias [52.76758938921129]
  We propose an effective bias-conflicting scoring method (ECS) to boost the identification accuracy. We also propose gradient alignment (GA) to balance the contributions of the mined bias-aligned and bias-conflicting samples. Experiments are conducted on multiple datasets in various settings, demonstrating that the proposed solution can mitigate the impact of unknown biases.
  arXiv  Detail & Related papers  (2023-02-22T14:50:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.