Statistical Software Engineering with Tuned Variables

• URL: http://arxiv.org/abs/2604.19822v1
• Date: Mon, 20 Apr 2026 09:15:27 GMT
• Title: Statistical Software Engineering with Tuned Variables
• Abstract summary: maintained artifact in an AI-enabled system is not code plus settings, but a versioned governed program space.<n>We argue that such choices should be treated as tuned variables: program variables maintained under governance as environments and evaluation sets evolve.
• Score: 0.5076419064097734
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The maintained artifact in an AI-enabled system is not code plus settings, but a versioned governed program space: domains, structural constraints, eligibility, evaluation assets, and a statistical release gate. AI-enabled systems operate under changing world conditions: provider models and APIs change, input distributions drift, evaluation sets age, and objectives such as quality, cost, latency, and safety are renegotiated over time. In practice, teams often respond through ad hoc changes to model choice, retrieval policy, prompt structure, and operational thresholds. Fixed-assignment reasoning is therefore insufficient: a chosen assignment is valid only relative to an environment, evaluation set, and policy state. We argue that such choices should be treated as tuned variables: program variables maintained under governance as environments and evaluation sets evolve. Building on SE4AI work and our prior work on governed tuning, this paper positions the governed space as the software-engineering object. Here, statistical means that promotion relies on sampled evaluation sets, estimated evidence, effect-size margins, and confidence/risk thresholds.

Related papers

• Reality Is the Final Verifier: On Two Key Gaps in Agentic Software Engineering [78.51705689800838]
  We propose an assurance-revision loop that uses deployment evidence to revise the requirements, model, or evaluator.<n>We then cast assured agentic development as a resource-allocation problem over human judgment, agent capability, and compute.
  arXiv  Detail & Related papers  (2026-09-10T17:58:48Z)
• TTHE: Test-Time Harness Evolution [50.26245555541721]
  Existing approaches optimize such harnesses before deployment, searching training or development data for a fixed agent workflow that is then frozen at test time.<n>We ask whether the harness can instead be optimized during evaluation itself, using only the unlabeled execution traces the agent produces on the test inputs.<n>During evaluation, TTHE maintains a population of candidate harnesses and refines them through an agentic proposer that reasons over their execution traces.<n>A judge then commits an improved harness from execution-derived proxy signals, and the selected program persists to govern subsequent inputs.
  arXiv  Detail & Related papers  (2026-07-09T05:53:39Z)
• A Stackelberg Framework for Resource-Aware LLM Agents: Learning, Repair, and Conditional Guarantees [1.9381445674403615]
  Large language model (LLM) agents increasingly operate as multi-turn systems must allocate context, prompt verbosity, and tool access under finite computational budgets.<n>We formulate resource governance as a contextual Stackelberg game: a controller commits to a quality target and a cost incentive, while an executor responds with resource actions over context, prompting, and tool usage.<n>We learn a conditional response model, optimize a leader policy against that model, and repair the resulting policy using real-API calibration and projection.
  arXiv  Detail & Related papers  (2026-06-22T08:38:22Z)
• An Agentic Retrieval Framework for Autonomous Context-Aware Data Quality Assessment [0.0]
  We propose a unified agentic-retrieval framework for autonomous context-aware data quality assessment.<n>The framework interprets natural-language descriptions of intended data usage, derives context-aware assessment strategies, and generates executable validation logic.<n>We implement the proposed framework as an end-to-end prototype and evaluate it across multiple usage scenarios applied to the same dataset.
  arXiv  Detail & Related papers  (2026-05-15T11:24:53Z)
• AI Harness Engineering: A Runtime Substrate for Foundation-Model Software Agents [1.4323566945483497]
  Foundation models have transformed automated code generation, yet autonomous software-engineering agents remain unreliable in realistic development settings.<n>We propose a model-harness-environment system in which a runtime substrate mediates how a foundation-model agent observes a project, acts on it, receives feedback, and establishes that a change is complete.<n>The framework reframes the central question of autonomous software engineering from whether a foundation model can produce a patch to whether the model-harness-environment system can produce a verifiably correct, attributed, and maintainable change.
  arXiv  Detail & Related papers  (2026-05-13T11:14:59Z)
• EmbodiedGovBench: A Benchmark for Governance, Recovery, and Upgrade Safety in Embodied Agent Systems [14.412476605788482]
  EmbodiedGovBench is a benchmark for governance-oriented evaluation of embodied agent systems.<n>It evaluates whether the system remains controllable, policy-bounded, recoverable, auditable, and evolution-safe under realistic perturbations.
  arXiv  Detail & Related papers  (2026-04-13T08:34:04Z)
• Position Paper: From Edge AI to Adaptive Edge AI [4.915029686150193]
  Edge AI is often framed as model compression and deployment under tight constraints.<n>We argue a stronger operational thesis: Edge AI in realistic deployments is necessarily adaptive.
  arXiv  Detail & Related papers  (2026-03-31T08:12:29Z)
• Quality-Driven Agentic Reasoning for LLM-Assisted Software Design: Questions-of-Thoughts (QoT) as a Time-Series Self-QA Chain [0.0]
  We introduce Questions-of-Thoughts (QoT), a quality-driven inference-time scaffold that turns a user goal into an ordered sequence of engineering steps.<n>We evaluate QoT across three representative backend engineering domains: API Design, Data Communication, and File Systems.
  arXiv  Detail & Related papers  (2026-03-10T23:49:09Z)
• Decoding ML Decision: An Agentic Reasoning Framework for Large-Scale Ranking System [26.405948122941467]
  We present GEARS, a framework that reframes ranking optimization as an autonomous discovery process.<n>We show that GEARS consistently identifies superior, near-Pareto-efficient policies by synergizing algorithmic signals with deep ranking context.
  arXiv  Detail & Related papers  (2026-02-20T22:24:01Z)
• AdaptNC: Adaptive Nonconformity Scores for Uncertainty-Aware Autonomous Systems in Dynamic Environments [7.201566646241765]
  Conformal Prediction methods maintain target coverage by adaptively scaling the conformal threshold.<n>We show that this fixed geometry leads to highly conservative, volume-inefficient prediction regions when environments undergo structural shifts.<n>We propose textbfAdaptNC, a framework for the joint online adaptation of both the nonconformity score parameters and the conformal threshold.
  arXiv  Detail & Related papers  (2026-02-02T04:41:35Z)
• Not All Preferences Are Created Equal: Stability-Aware and Gradient-Efficient Alignment for Reasoning Models [52.48582333951919]
  We propose a dynamic framework designed to enhance alignment reliability by maximizing the Signal-to-Noise Ratio of policy updates.<n>SAGE (Stability-Aware Gradient Efficiency) integrates a coarse-grained curriculum mechanism that refreshes candidate pools based on model competence.<n> Experiments on multiple mathematical reasoning benchmarks demonstrate that SAGE significantly accelerates convergence and outperforms static baselines.
  arXiv  Detail & Related papers  (2026-02-01T12:56:10Z)
• OmniQuality-R: Advancing Reward Models Through All-Encompassing Quality Assessment [55.59322229889159]
  We propose OmniQuality-R, a unified reward modeling framework that transforms multi-task quality reasoning into continuous and interpretable reward signals.<n>We use a reasoning-enhanced reward modeling dataset to form a reliable chain-of-thought dataset for supervised fine-tuning.<n>We evaluate OmniQuality-R on three key IQA tasks: aesthetic quality assessment, technical quality evaluation, and text-image alignment.
  arXiv  Detail & Related papers  (2025-10-12T13:46:28Z)
• SKADA-Bench: Benchmarking Unsupervised Domain Adaptation Methods with Realistic Validation On Diverse Modalities [50.6382396309597]
  Unsupervised Domain Adaptation (DA) consists of adapting a model trained on a labeled source domain to perform well on an unlabeled target domain with some data distribution shift.<n>We present a complete and fair evaluation of existing shallow algorithms, including reweighting, mapping, and subspace alignment.<n>Our benchmark highlights the importance of realistic validation and provides practical guidance for real-life applications.
  arXiv  Detail & Related papers  (2024-07-16T12:52:29Z)
• Off-Policy Evaluation for Large Action Spaces via Policy Convolution [60.6953713877886]
  Policy Convolution family of estimators uses latent structure within actions to strategically convolve the logging and target policies. Experiments on synthetic and benchmark datasets demonstrate remarkable mean squared error (MSE) improvements when using PC.
  arXiv  Detail & Related papers  (2023-10-24T01:00:01Z)
• Position: AI Evaluation Should Learn from How We Test Humans [65.36614996495983]
  We argue that psychometrics, a theory originating in the 20th century for human assessment, could be a powerful solution to the challenges in today's AI evaluations.
  arXiv  Detail & Related papers  (2023-06-18T09:54:33Z)
• Temporal-Difference Value Estimation via Uncertainty-Guided Soft Updates [110.92598350897192]
  Q-Learning has proven effective at learning a policy to perform control tasks. estimation noise becomes a bias after the max operator in the policy improvement step. We present Unbiased Soft Q-Learning (UQL), which extends the work of EQL from two action, finite state spaces to multi-action, infinite state Markov Decision Processes.
  arXiv  Detail & Related papers  (2021-10-28T00:07:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.