SAHOO: Safeguarded Alignment for High-Order Optimization Objectives in Recursive Self-Improvement

• URL: http://arxiv.org/abs/2603.06333v1
• Date: Fri, 06 Mar 2026 14:44:51 GMT
• Title: SAHOO: Safeguarded Alignment for High-Order Optimization Objectives in Recursive Self-Improvement
• Abstract summary: SAHOO is a framework to monitor and control drift through three safeguards.<n>GDI is a multi-signal detector combining semantic, lexical, structural, and distributional measures.<n> constraint preservation checks enforce safety-critical invariants such as syntactic correctness and non-hallucination.
• Score: 16.505918019260964
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recursive self-improvement is moving from theory to practice: modern systems can critique, revise, and evaluate their own outputs, yet iterative self-modification risks subtle alignment drift. We introduce SAHOO, a practical framework to monitor and control drift through three safeguards: (i) the Goal Drift Index (GDI), a learned multi-signal detector combining semantic, lexical, structural, and distributional measures; (ii) constraint preservation checks that enforce safety-critical invariants such as syntactic correctness and non-hallucination; and (iii) regression-risk quantification to flag improvement cycles that undo prior gains. Across 189 tasks in code generation, mathematical reasoning, and truthfulness, SAHOO produces substantial quality gains, including 18.3 percent improvement in code tasks and 16.8 percent in reasoning, while preserving constraints in two domains and maintaining low violations in truthfulness. Thresholds are calibrated on a small validation set of 18 tasks across three cycles. We further map the capability-alignment frontier, showing efficient early improvement cycles but rising alignment costs later and exposing domain-specific tensions such as fluency versus factuality. SAHOO therefore makes alignment preservation during recursive self-improvement measurable, deployable, and systematically validated at scale.

Related papers

• Simultaneous Coverage and Efficiency Guarantee in Online Conformal Prediction [5.533296779866529]
  Adaptive conformal inference (ACI) of Gibbs and Cands and its variants are the standard approach to online conformal prediction under distribution shift.<n>Existing guarantees say nothing about prediction-set size, so validity can be achieved trivially at the cost of unduly wide prediction sets.<n>We consider a unified online learning framework that simultaneously controls absolute, non-optimal coverage violation and prediction-set efficiency.
  arXiv  Detail & Related papers  (2026-07-29T07:56:59Z)
• AdaExplore: Failure-Driven Adaptation and Diversity-Preserving Search for Efficient Kernel Generation [59.964989458924585]
  AdaExplore is an agent framework that enables self-improvement via accumulated execution feedback for performance-critical kernel code generation.<n>In the adaptation stage, the agent synthesizes tasks and converts recurring failures into a reusable memory of validity rules.<n>In the search stage, the agent organizes candidate kernels as a tree and alternates between small local refinements and larger structural regeneration.
  arXiv  Detail & Related papers  (2026-04-17T18:25:03Z)
• Less Approximates More: Harmonizing Performance and Confidence Faithfulness via Hybrid Post-Training for High-Stakes Tasks [52.81286869496811]
  Large language models are increasingly deployed in high-stakes tasks, where confident yet incorrect inferences may cause real-world harm.<n>We propose to jointly optimize unsupervised Reinforcement Learning from Internal Feedback with reasoning-trace-guided Reasoning Distillation.<n>We introduce HyTuning, a hybrid post-training framework that adaptively reweights RD and RLIF via a PRG-style metric.
  arXiv  Detail & Related papers  (2026-04-09T16:50:11Z)
• WARP: Guaranteed Inner-Layer Repair of NLP Transformers [4.191577542171072]
  We present WARP, a constraint-based repair framework that extends repair beyond the last layer of Transformer models.<n> WARP formulates repair as a convex quadratic program derived from a first-order linearization of the logit gap.<n>We show that WARP converges to solutions satisfying all repair constraints under mild assumptions.
  arXiv  Detail & Related papers  (2026-04-01T14:12:49Z)
• Taming the Instability: A Robust Second-Order Optimizer for Federated Learning over Non-IID Data [77.19117648492104]
  We present Federated Robust Curvature Optimization (FedRCO), a novel second-order optimization framework designed to improve convergence speed and reduce communication cost.<n>FedRCO addresses these challenges by integrating an efficient approximate curvature with a provable stability mechanism.<n>We show that FedRCO achieves superior robustness against diverse non-IID scenarios while achieving higher accuracy and faster convergence than both state-of-the-art first-order and second-order methods.
  arXiv  Detail & Related papers  (2026-03-30T11:37:46Z)
• Distilling Deep Reinforcement Learning into Interpretable Fuzzy Rules: An Explainable AI Framework [1.9180507495574395]
  Deep Reinforcement Learning (DRL) agents achieve remarkable performance in continuous control but remain opaque, hindering deployment in safety-critical domains.<n>Existing explainability methods either provide only local insights (SHAP, LIME) or employ oversimplified surrogates failing to capture continuous dynamics.<n>This work proposes a Fuzzy System (FCS) distilling neural policies into human-readable IF-THEN rules through K-Means clustering for state-readable partitioning.<n>Three quantifiable metrics are introduced: Rule Density (FRAD) measuring explanation focus, Fuzzy Set Coverage (FSC) validating vocabulary completeness, and Action Space Granularity (
  arXiv  Detail & Related papers  (2026-02-24T23:53:01Z)
• Constraint-Rectified Training for Efficient Chain-of-Thought [60.52883907721588]
  Chain-of-Thought (CoT) has significantly enhanced the reasoning capabilities of Large Language Models (LLMs)<n>While longer reasoning traces can improve answer quality and unlock abilities such as self-correction, they also incur high inference costs and often introduce redundant steps, known as overthinking.<n>Recent research seeks to develop efficient reasoning strategies that balance reasoning length and accuracy.
  arXiv  Detail & Related papers  (2026-02-13T02:13:45Z)
• Unifying Stable Optimization and Reference Regularization in RLHF [64.16830602324345]
  This paper introduces a unified regularization approach that balances objectives of preventing reward hacking and maintaining stable policy updates.<n>Our simple yet principled alignment objective yields a weighted supervised fine-tuning loss with a superior trade-off, which demonstrably improves both alignment results and implementation complexity.
  arXiv  Detail & Related papers  (2026-02-12T03:31:19Z)
• Causality-Inspired Safe Residual Correction for Multivariate Time Series [12.183024727781449]
  We propose CRC (Causality-inspired Safe Residual Correction), a plug-and-play framework explicitly designed to ensure non-degradation.<n>It employs a causality-inspired encoder to expose direction-aware structure by decoupling self- and cross-variable dynamics, and a hybrid corrector to model residual errors.<n>Experiments show that CRC consistently improves accuracy, while an in-depth ablation study confirms that its core safety mechanisms ensure exceptionally high non-degradation rates (NDR)
  arXiv  Detail & Related papers  (2025-12-27T01:34:14Z)
• GRPO-Guard: Mitigating Implicit Over-Optimization in Flow Matching via Regulated Clipping [63.33669214116784]
  GRPO-Guard is a simple yet effective enhancement to existing GRPO frameworks.<n>It restores a balanced and step-consistent importance ratio, ensuring that PPO clipping properly constrains harmful updates.<n>It substantially mitigates implicit over-optimization without relying on heavy KL regularization.
  arXiv  Detail & Related papers  (2025-10-25T14:51:17Z)
• SGM: A Statistical Godel Machine for Risk-Controlled Recursive Self-Modification [24.250783705030653]
  We introduce the Statistical Godel Machine (SGM), the first statistical architecture for safety edits.<n>SGM replaces proof-based requirements with statistical confidence tests (e-values, Hoeffding bounds), admitting a modification only when superiority is certified at a chosen confidence level.<n>We also propose Confirm-Triggered Harmonic Spending (CTHS), which indexes spending by confirmation events rather than rounds, concentrating the error budget on promising edits.
  arXiv  Detail & Related papers  (2025-10-11T14:09:37Z)
• The Alignment Auditor: A Bayesian Framework for Verifying and Refining LLM Objectives [8.030821324147515]
  Inverse Reinforcement Learning can infer reward functions from behaviour.<n>Existing approaches either produce a single, overconfident reward estimate or fail to address the fundamental ambiguity of the task.<n>This paper introduces a principled auditing framework that re-frames reward inference from a simple estimation task to a comprehensive process for verification.
  arXiv  Detail & Related papers  (2025-10-07T16:25:14Z)
• Boundary-to-Region Supervision for Offline Safe Reinforcement Learning [56.150983204962735]
  Boundary-to-Region (B2R) is a framework that enables asymmetric conditioning through cost signal realignment.<n>B2R redefines CTG as a boundary constraint under a fixed safety budget, unifying the cost distribution of all feasible trajectories.<n> Experimental results show that B2R satisfies safety constraints in 35 out of 38 safety-critical tasks.
  arXiv  Detail & Related papers  (2025-09-30T03:38:20Z)
• Adaptive Dual Uncertainty Optimization: Boosting Monocular 3D Object Detection under Test-Time Shifts [80.32933059529135]
  Test-Time Adaptation (TTA) methods have emerged to adapt to target distributions during inference.<n>We propose Dual Uncertainty Optimization (DUO), the first TTA framework designed to jointly minimize both uncertainties for robust M3OD.<n>In parallel, we design a semantic-aware normal field constraint that preserves geometric coherence in regions with clear semantic cues.
  arXiv  Detail & Related papers  (2025-08-28T07:09:21Z)
• Core Safety Values for Provably Corrigible Agents [2.6451153531057985]
  We introduce the first implementable framework for corrigibility, with provable guarantees in multi-step, partially observed environments.<n>Our framework replaces a single reward with five *structurally separate* utility heads.<n>For open-ended settings where adversaries can modify the agent, we prove that deciding whether an arbitrary post-hack agent will ever violate corrigibility is undecidable.
  arXiv  Detail & Related papers  (2025-07-28T16:19:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.