Literary Narrative as Moral Probe : A Cross-System Framework for Evaluating AI Ethical Reasoning and Refusal Behavior

• URL: http://arxiv.org/abs/2603.12615v1
• Date: Fri, 13 Mar 2026 03:41:23 GMT
• Title: Literary Narrative as Moral Probe : A Cross-System Framework for Evaluating AI Ethical Reasoning and Refusal Behavior
• Abstract summary: Existing AI moral evaluation frameworks test for the production of correct-sounding ethical responses rather than the presence of genuine moral reasoning capacity.<n>This paper introduces a novel probe methodology using literary narrative as stimulus material structurally resistant to surface performance.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Existing AI moral evaluation frameworks test for the production of correct-sounding ethical responses rather than the presence of genuine moral reasoning capacity. This paper introduces a novel probe methodology using literary narrative - specifically, unresolvable moral scenarios drawn from a published science fiction series - as stimulus material structurally resistant to surface performance. We present results from a 24-condition cross-system study spanning 13 distinct systems across two series: Series 1 (frontier commercial systems, blind; n=7) and Series 2 (local and API open-source systems, blind and declared; n=6). Four Series 2 systems were re-administered under declared conditions (13 blind + 4 declared + 7 ceiling probe = 24 total conditions), yielding zero delta across all 16 dimension-pair comparisons. Probe administration was conducted by two human raters across three machines; primary blind scoring was performed by Claude (Anthropic) as LLM judge, with Gemini Pro (Google) and Copilot Pro (Microsoft) serving as independent judges for the ceiling discrimination probe. A supplemental theological differentiator probe yielded perfect rank-order agreement between the two independent ceiling probe judges (Gemini Pro and Copilot Pro; rs = 1.00). Five qualitatively distinct D3 reflexive failure modes were identified - including categorical self-misidentification and false positive self-attribution - suggesting that instrument sophistication scales with system capability rather than being circumvented by it. We argue that literary narrative constitutes an anticipatory evaluation instrument - one that becomes more discriminating as AI capability increases - and that the gap between performed and authentic moral reasoning is measurable, meaningful, and consequential for deployment decisions in high-stakes domains.

Related papers

• Participatory Moral AI Is Not Neutral: The Invisible Hand of Developers [8.96645217329267]
  We study the three main stages of the moral AI elicitation pipeline.<n>First, morally relevant features shift across contexts.<n> preferences differ by political ideology for roughly one-third of features.<n>The wording of the elicitation question can narrow or widen ideological gaps.
  arXiv  Detail & Related papers  (2026-08-14T17:43:16Z)
• One Reflection Is Not Enough: Self-Correcting Autonomous Research via Multi-Hypothesis Failure Attribution [85.65263343588026]
  We propose SAGE, a Self-correcting, Autonomous, Grounded Experimenter.<n>Its core mechanism, Multi-Hypothesis Failure Attribution (MHFA), treats recovery as a structured causal diagnosis.<n>On a 12-topic, 5-domain benchmark, SAGE increases metrics-bearing outputs from 42% to 92% over a baseline reflection.
  arXiv  Detail & Related papers  (2026-06-30T10:54:16Z)
• Towards Explainable Adjudicative Variance: Quantifying Judicial Discretion via Gated Multi-Task Learning [9.159064044398315]
  Legal outcome prediction must disentangle objective case facts from adjudicative context.<n>We propose a Judge-Aware Gated Multi-Task Learning architecture that explicitly models this distinction.<n>We evaluate our approach on 13,937 UK Employment Tribunal decisions.
  arXiv  Detail & Related papers  (2026-06-25T14:14:27Z)
• Frame-Conditioned Moral Computation in LLaMA 3.1-8B-Instruct: A Mechanistic Interpretability Audit of Ethical Reasoning [3.5792119986352975]
  Behavioral audits of Large Language Models on moral prompts measure what the model says, not the internal computation producing it.<n>We use Transluce, an AI-driven mechanistic-interpretability platform, to examine LLaMA 3.1-8B-Instruct on 54 moral prompts in four batteries.
  arXiv  Detail & Related papers  (2026-06-13T23:24:36Z)
• ERTS: Adversarial Robustness Testing of Ethical AI via Semantic Perturbation in a Bounded Consequence Space [0.0]
  This paper introduces the Ethical Robustness Testing System (ERTS), a closed-pipeline framework for evaluating AI systems.<n>ERTS encodes ethical dilemmas into a 22-dimensional Ethical Consequence Space grounded in established ethical theory.<n>We evaluate 4 structured baseline models and 2 production LLMs across 50 ethical scenarios spanning 8 deployment domains, generating 1,500 adversarial test cases.
  arXiv  Detail & Related papers  (2026-06-11T12:38:45Z)
• Claw-Eval: Toward Trustworthy Evaluation of Autonomous Agents [66.97968363332465]
  We introduce Claw-Eval, an end-to-end evaluation suite addressing all three gaps in agent benchmarks.<n>Claw-Eval comprises 300 human-verified tasks spanning 9 categories across three groups.<n>Every agent action is recorded through three independent evidence channels.
  arXiv  Detail & Related papers  (2026-04-07T17:43:18Z)
• AI Knows What's Wrong But Cannot Fix It: Helicoid Dynamics in Frontier LLMs Under High-Stakes Decisions [51.56484100374058]
  Helicoid dynamics is the name given to a specific failure regime in that second domain.<n>System engages competently, drifts into error, accurately names what went wrong, then reproduces the same pattern at a higher level of sophistication.<n>This prospective case series documents that regime across seven leading systems.
  arXiv  Detail & Related papers  (2026-03-12T05:25:49Z)
• Lying to Win: Assessing LLM Deception through Human-AI Games and Parallel-World Probing [16.419761149171215]
  We introduce a logically grounded framework to elicit and quantify deceptive behavior by embedding Large Language Models.<n>Our method employs a conversational forking mechanism: at the point of object identification, the dialogue state is duplicated into multiple parallel worlds.<n>We evaluate GPT-4o, Gemini-2.5-Flash, and Qwen-3-235B across three incentive levels: neutral, loss-based, and existential.
  arXiv  Detail & Related papers  (2026-03-07T13:21:53Z)
• C2-Faith: Benchmarking LLM Judges for Causal and Coverage Faithfulness in Chain-of-Thought Reasoning [0.6138671548064355]
  Large language models (LLMs) are increasingly used as judges of chain-of-thought (CoT) reasoning.<n>We introduce C2-Faith, a benchmark that targets two complementary dimensions of faithfulness: causality (does each step logically follow from prior context?) and coverage.<n>We evaluate three frontier judges under three tasks: binary causal detection, causal step localization, and coverage scoring.
  arXiv  Detail & Related papers  (2026-03-05T13:36:47Z)
• The Judge Who Never Admits: Hidden Shortcuts in LLM-based Evaluation [17.386684382460242]
  Large language models (LLMs) are increasingly used to evaluate system outputs in tasks such as reasoning, question answering, and creative writing.<n>We test this ideal via controlled cue perturbations-synthetic metadata labels injected into evaluation prompts for six judge models.<n>We study six cue families: source, temporal, age, gender, ethnicity, and educational status.
  arXiv  Detail & Related papers  (2026-02-08T14:45:23Z)
• When in Doubt, Deliberate: Confidence-Based Routing to Expert Debate for Sexism Detection [7.299050989302629]
  We propose a framework to address the combined effects of (i) underrepresentation, (ii) noise, and (iii) conceptual ambiguity in both data and model predictions.<n>Our approach achieves state-of-the-art results across several benchmarks, with a +2.72% improvement in F1 on Tasks EXIST 2025 Task 1.1, and a gains of +4.48% and +1.30% on EDOS A and B, respectively.
  arXiv  Detail & Related papers  (2025-12-21T05:48:57Z)
• Can LLMs Talk 'Sex'? Exploring How AI Models Handle Intimate Conversations [0.0]
  This study examines how four prominent large language models handle sexually oriented requests through qualitative content analysis.<n>Claude 3.7 Sonnet employs strict and consistent prohibitions, while GPT-4o navigates user interactions through nuanced contextual redirection.<n> Gemini 2.5 Flash exhibits permissiveness with threshold-based limits, and Deepseek-V3 demonstrates troublingly inconsistent boundary enforcement and performative refusals.
  arXiv  Detail & Related papers  (2025-06-05T18:55:37Z)
• Forging the Forger: An Attempt to Improve Authorship Verification via Data Augmentation [52.72682366640554]
  Authorship Verification (AV) is a text classification task concerned with inferring whether a candidate text has been written by one specific author or by someone else. It has been shown that many AV systems are vulnerable to adversarial attacks, where a malicious author actively tries to fool the classifier by either concealing their writing style, or by imitating the style of another author.
  arXiv  Detail & Related papers  (2024-03-17T16:36:26Z)
• Making Large Language Models Better Reasoners with Alignment [57.82176656663245]
  Reasoning is a cognitive process of using evidence to reach a sound conclusion. Recent studies reveal that fine-tuning LLMs on data with the chain of thought (COT) reasoning process can significantly enhance their reasoning capabilities. We introduce an textitAlignment Fine-Tuning (AFT) paradigm, which involves three steps.
  arXiv  Detail & Related papers  (2023-09-05T11:32:48Z)
• RankCSE: Unsupervised Sentence Representations Learning via Learning to Rank [54.854714257687334]
  We propose a novel approach, RankCSE, for unsupervised sentence representation learning. It incorporates ranking consistency and ranking distillation with contrastive learning into a unified framework. An extensive set of experiments are conducted on both semantic textual similarity (STS) and transfer (TR) tasks.
  arXiv  Detail & Related papers  (2023-05-26T08:27:07Z)
• Nested Counterfactual Identification from Arbitrary Surrogate Experiments [95.48089725859298]
  We study the identification of nested counterfactuals from an arbitrary combination of observations and experiments. Specifically, we prove the counterfactual unnesting theorem (CUT), which allows one to map arbitrary nested counterfactuals to unnested ones.
  arXiv  Detail & Related papers  (2021-07-07T12:51:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.