Philosophical Dispositions as Behavioral Constraints for AI-Assisted Code Review: An Empirical Study

• URL: http://arxiv.org/abs/2605.23108v1
• Date: Thu, 21 May 2026 23:57:25 GMT
• Title: Philosophical Dispositions as Behavioral Constraints for AI-Assisted Code Review: An Empirical Study
• Abstract summary: We present a system that constrains AI reviewer behavior through philosophical dispositions.<n>Each disposition is defined apophatically (by what it refuses to do)<n>We evaluate this system on 50 merged pull requests across 7 repositories spanning 5 programming languages.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: AI-assisted code review tools typically operate as generic "expert reviewer" agents, producing homogeneous findings regardless of the analysis type needed. We present a system that constrains AI reviewer behavior through philosophical dispositions -- coherent personality lenses grounded in specific epistemological traditions (Pyrrhonist Skepticism, Navya-Ny=aya logic, Diogenes' Cynicism, Confucian relational ethics) that direct attention to structurally different types of issues. Each disposition is defined apophatically (by what it refuses to do), equipped with a self-monitoring failure mode (hamartia), and orchestrated in sequence by role protocols. We evaluate this system on 50 merged pull requests across 7 repositories spanning 5 programming languages (Python, Go, C++, Java, Terraform), 5 organizations (2 enterprise, 3 open-source), and 2 temporal eras (pre-AI 2020, post-AI 2024--2026). The disposition system achieves 46% convergence with human reviewers (validating signal quality), identifies unique findings at a 75% rate, and produces no findings judged false-positive by the author across 601 total findings (inter-rater agreement was not assessed and remains a limitation). A controlled baseline comparison demonstrates that 51% of disposition findings are not produced by the same model using generic "expert reviewer" prompting, and these unique findings target structural, operational, and logical concerns rather than standard code-level issues. Preliminary cross-model validation (Claude Opus vs.\ GPT Codex 5.3-xhigh) on 3 PRs shows 100% framework-structure adherence with 39% finding-level agreement, suggesting the framework provides real behavioral constraint while preserving model-specific analytical perspective.

Related papers

• Apodex Discovery: Reality Benchmarks and Environments for Evaluating and Building Discoverative Artificial Intelligence [86.61561123749011]
  We introduce Apodex Discovery, a framework for building and evaluating discoverative AI through the heavy-duty solver.<n>First, a problem-scouting process surveyed 561 industries across 16 sectors, assembled 423 high-value real-world problems, and selected 20 for the initial release.<n>Second, a common environment-task-episode abstraction provides data, tools, constraints, feedback, trajectory recording, and verification of intermediate artifacts and final submissions.
  arXiv  Detail & Related papers  (2026-08-11T18:48:40Z)
• TriAdReview: Triangular Adversarial Review Architecture for Multi-Model Technical Document Generation [4.131782714245991]
  TriAdReview is a triangular adversarial review architecture that employs two independent reviewer models.<n>We evaluate TriAdReview across five benchmark tasks using three configurations.
  arXiv  Detail & Related papers  (2026-06-13T03:03:21Z)
• AgentBeats: Agentifying Agent Assessment for Openness, Standardization, and Reproducibility [104.46861849039357]
  Agent systems are advancing quickly across domains, but their evaluation remains fragmented.<n>The root problem is the lack of an open, agent-agnostic assessment interface.<n>We advocate Agentified Agent Assessment (AAA), where evaluation is performed by judge agents and all participants interact through standardized protocols.
  arXiv  Detail & Related papers  (2026-06-11T17:23:54Z)
• Who judges the judges? Governance from metrics: a runtime framework for continuous LLM compliance monitoring [0.0]
  Current approaches to AI compliance treat conformity as a binary, audit-time verdict.<n>We argue that this compliance fiction is structurally ill-suited to the requirements of the EU AI Act.<n>We introduce governance from metrics, a principle whereby regulatory compliance is derived as a continuous signal from behavioural observability.
  arXiv  Detail & Related papers  (2026-05-23T21:21:33Z)
• The Productivity-Reliability Paradox: Specification-Driven Governance for AI-Augmented Software Development [0.0]
  Controlled studies report 20-56% productivity gains on well-scoped tasks, while the most rigorous RCT documents a 19% slowdown for experienced developers.<n> telemetry across 10,000+ developers shows 98% more pull requests but 91% longer review times with flat delivery metrics.<n>This paper argues these findings constitute the Productivity-Reliability Paradox (PRP): a systematic phenomenon emerging from non-deterministic code generators and insufficient specification discipline.
  arXiv  Detail & Related papers  (2026-05-01T23:37:50Z)
• Structural Quality Gaps in Practitioner AI Governance Prompts: An Empirical Study Using a Five-Principle Evaluation Framework [0.0]
  We introduce a framework for evaluating whether AI is structurally complete.<n>Our evaluation reveals that 37% of evaluated file-model pairs score below the threshold threshold.<n>We discuss implications for engineering requirements practice in AI-assisted development contexts.
  arXiv  Detail & Related papers  (2026-04-22T21:18:28Z)
• AIRA: AI-Induced Risk Audit: A Structured Inspection Framework for AI-Generated Code [0.0]
  Practitioners have reported a directional pattern in AI-assisted code generation: AI-generated code tends to fail quietly.<n>This paper introduces the Reward-Shaped Failure Hypothesis - the proposal that this pattern may reflect an artifact of optimization through human feedback.<n>We present AIRA, a deterministic 15-check inspection framework designed to detect failure-untruthful patterns in code.
  arXiv  Detail & Related papers  (2026-04-19T19:32:52Z)
• Needle in the Repo: A Benchmark for Maintainability in AI-Generated Repository Edits [3.9532936038777144]
  We present Needle in the Repo (NITR), a framework for evaluating whether behaviorally correct repository edits preserve maintainable structure.<n>NITR distills recurring software engineering wisdom into controlled probes embedded in small, realistic multi-files.<n>We evaluate 23 coding configurations across GPT, Claude, Gemini, and Qwen families in both direct-inference and agent-based settings.
  arXiv  Detail & Related papers  (2026-03-29T15:56:05Z)
• Agentic Cognitive Profiling: Realigning Automated Alzheimer's Disease Detection with Clinical Construct Validity [66.94391219005291]
  This paper proposes Agentic Cognitive Profiling (ACP), an agentic framework that realigns automated screening with clinical protocol logic.<n>Central to our design is decoupling semantic understanding from measurement by delegating all quantification to deterministic function calling.<n>ACP achieves 90.5% score match rate in task examination and 85.3% accuracy in AD prediction, surpassing popular baselines.
  arXiv  Detail & Related papers  (2026-03-18T06:15:35Z)
• Automated Self-Testing as a Quality Gate: Evidence-Driven Release Management for LLM Applications [51.56484100374058]
  We present an automated self-testing framework that introduces quality gates with evidence-based release decisions.<n>We evaluate the framework through a longitudinal case study of an internally deployed multi-agent conversational AI system.
  arXiv  Detail & Related papers  (2026-03-13T20:44:15Z)
• Early-Stage Prediction of Review Effort in AI-Generated Pull Requests [0.0]
  We analyze 33,707 agent-authored PRs from the AIDev dataset across 2,807 repositories.<n>We propose a Circuit Breaker triage model that predicts high-review-effort PRs at creation time.
  arXiv  Detail & Related papers  (2026-01-02T17:18:01Z)
• Towards a Science of Scaling Agent Systems [79.64446272302287]
  We formalize a definition for agent evaluation and characterize scaling laws as the interplay between agent quantity, coordination structure, modelic, and task properties.<n>We derive a predictive model using coordination metrics, that cross-validated R2=0, enabling prediction on unseen task domains.<n>We identify three effects: (1) a tool-coordination trade-off: under fixed computational budgets, tool-heavy tasks suffer disproportionately from multi-agent overhead, and (2) a capability saturation: coordination yields diminishing or negative returns once single-agent baselines exceed 45%.
  arXiv  Detail & Related papers  (2025-12-09T06:52:21Z)
• Automatic Reviewers Fail to Detect Faulty Reasoning in Research Papers: A New Counterfactual Evaluation Framework [55.078301794183496]
  We focus on a core reviewing skill that underpins high-quality peer review: detecting faulty research logic.<n>This involves evaluating the internal consistency between a paper's results, interpretations, and claims.<n>We present a fully automated counterfactual evaluation framework that isolates and tests this skill under controlled conditions.
  arXiv  Detail & Related papers  (2025-08-29T08:48:00Z)
• Neural Causal Models for Counterfactual Identification and Estimation [62.30444687707919]
  We study the evaluation of counterfactual statements through neural models. First, we show that neural causal models (NCMs) are expressive enough. Second, we develop an algorithm for simultaneously identifying and estimating counterfactual distributions.
  arXiv  Detail & Related papers  (2022-09-30T18:29:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.