SHARP: A Self-Evolving Human-Auditable Rubric Policy for Financial Trading Agents

• URL: http://arxiv.org/abs/2605.06822v1
• Date: Thu, 07 May 2026 18:23:44 GMT
• Title: SHARP: A Self-Evolving Human-Auditable Rubric Policy for Financial Trading Agents
• Abstract summary: Large language models (LLMs) are increasingly deployed for autonomous financial trading.<n>We introduce the Self-Evolving Human-Auditable Policy (SHARP), a neuro-symbolic framework that replaces unconstrained text mutation with structured, symbolic policy optimization.
• Score: 11.616028403813031
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language models (LLMs) are increasingly deployed for autonomous financial trading, a domain requiring continuous adaptation to noisy, non-stationary markets. Existing self-improving agents typically address this through unbounded free-form prompt optimization. However, in low signal-to-noise environments with delayed scalar rewards (P\&L), this unstructured approach exacerbates the fundamental credit assignment problem: optimizers cannot reliably distinguish systematic logic flaws from stochastic market variance, inevitably leading to policy drift. To overcome this bottleneck, we introduce the Self-Evolving Human-Auditable Rubric Policy (SHARP), a neuro-symbolic framework that replaces unconstrained text mutation with structured, symbolic policy optimization. SHARP confines the agent's reasoning to a bounded, human-readable rubric of explicit condition-action rules. When sub-optimal trades occur, an attribution agent employs cross-sample reasoning across multiple samples to isolate specific rule failures. This enables targeted, atomic policy edits that are subsequently regularized through strict walk-forward validation. Evaluated across three diverse equity sectors and four LLM backbones, SHARP consistently transforms generic initial heuristics into highly robust strategies, lifting the empirical performance of compact models by 10 to 20 percentage points on average (e.g., GPT-4o-mini). Ultimately, SHARP demonstrates that LLMs can achieve dynamic and efficient adaptation while significantly enhancing the structural transparency and auditability demanded by institutional finance.

Related papers

• Ratio-Variance Regularized Policy Optimization [64.95520246570446]
  We show that explicitly constraining the policy ratio variance provides a principled local approximation to trust-region constraints.<n>We introduce $bf R2bf VPO$ (Ratio-Variance Regularized Policy Optimization), which implements this constraint via a primal-dual optimization framework.
  arXiv  Detail & Related papers  (2026-05-26T09:53:42Z)
• One-Way Policy Optimization for Self-Evolving LLMs [63.8638342097375]
  Reinforcement Learning with Verifiable Rewards (RLVR) has become a promising paradigm for scaling reasoning capabilities of Large Language Models (LLMs)<n>We propose One-Way Policy Optimization (OWPO), a method based on the principle of decoupling optimization direction from update magnitude.<n> Experimental results demonstrate that OWPO outperforms strong baselines, including DAPO, OPD, and MOPD.
  arXiv  Detail & Related papers  (2026-05-21T08:25:27Z)
• Reliability and Effectiveness of Autonomous AI Agents in Supply Chain Management [15.19525853427301]
  This paper studies autonomous generative AI agents in multi-echelon supply chains using the MIT Beer Game.<n>We identify four inference-time levers that shape performance: model selection, policies and guardrails, centralized data sharing, and prompt engineering.
  arXiv  Detail & Related papers  (2026-05-16T15:11:35Z)
• AgenticAITA: A Proof-Of-Concept About Deliberative Multi-Agent Reasoning for Autonomous Trading Systems [0.0]
  AGENTICAITA is an agentic AI framework that replaces the traditional signal then execute paradigm with a fully autonomous loop.<n>The framework demonstrates operational correctness of the deliberative pipeline, achieving 157 zero-intervention invocations across 76 assets.
  arXiv  Detail & Related papers  (2026-05-01T16:25:43Z)
• Robust Regularized Policy Iteration under Transition Uncertainty [6.7431287237221085]
  We formulate offline RL as robust policy optimization, treating the transition kernel as a decision variable within an uncertainty set.<n>We propose Robust Regularized Policy Iteration (RRPI), which replaces the intractable max-min bilevel objective with a tractable KL-regularized surrogate.<n>Experiments on D4RL benchmarks demonstrate that RRPI achieves strong average performance, outperforming recent baselines.
  arXiv  Detail & Related papers  (2026-03-10T08:18:27Z)
• Improving Search Agent with One Line of Code [68.58667107354253]
  Tool-based Agentic Reinforcement Learning (TARL) has emerged as a promising paradigm for training search agents to interact with external tools.<n>We propose textbfSearch textbfAgent textbfPolicy textbfOptimization (textbfSAPO), which stabilizes training via a conditional token-level KL constraint.
  arXiv  Detail & Related papers  (2026-03-10T04:07:39Z)
• Socially-Weighted Alignment: A Game-Theoretic Framework for Multi-Agent LLM Systems [17.658093330392052]
  We propose a game-theoretic framework that modifies inference-time decision making by interpolating between an agent's private objective and an estimate of group welfare.<n>We show that SWA induces a critical threshold $*=(n-)/(n-1)$ above which agents no longer have marginal incentive to increase demand under overload.
  arXiv  Detail & Related papers  (2026-02-16T05:17:58Z)
• Rethinking the Trust Region in LLM Reinforcement Learning [72.25890308541334]
  Proximal Policy Optimization (PPO) serves as the de facto standard algorithm for Large Language Models (LLMs)<n>We propose Divergence Proximal Policy Optimization (DPPO), which substitutes clipping with a more principled constraint.<n>DPPO achieves superior training and efficiency compared to existing methods, offering a more robust foundation for RL-based fine-tuning.
  arXiv  Detail & Related papers  (2026-02-04T18:59:04Z)
• MAESTRO: Meta-learning Adaptive Estimation of Scalarization Trade-offs for Reward Optimization [56.074760766965085]
  Group-Relative Policy Optimization has emerged as an efficient paradigm for aligning Large Language Models (LLMs)<n>We propose MAESTRO, which treats reward scalarization as a dynamic latent policy, leveraging the model's terminal hidden states as a semantic bottleneck.<n>We formulate this as a contextual bandit problem within a bi-level optimization framework, where a lightweight Conductor network co-evolves with the policy by utilizing group-relative advantages as a meta-reward signal.
  arXiv  Detail & Related papers  (2026-01-12T05:02:48Z)
• Ratio-Variance Regularized Policy Optimization for Efficient LLM Fine-tuning [48.34492357368989]
  We propose a primal-dual framework that supports stable on-policy learning and enables principled off-policy data reuse.<n>$R2VPO$ achieves superior performance with average relative gains of up to 17% over strong clipping-based baselines.
  arXiv  Detail & Related papers  (2026-01-06T14:01:42Z)
• Soft Adaptive Policy Optimization [67.61886077470528]
  Reinforcement learning plays an increasingly important role in enhancing the reasoning capabilities of large language models.<n>Existing group-based policy optimization methods, such as GSPO and GRPO, alleviate this problem via hard clipping.<n>We propose Soft Adaptive Policy Optimization (SAPO), which replaces hard clipping with a smooth, temperature-controlled gate.
  arXiv  Detail & Related papers  (2025-11-25T14:25:19Z)
• Robust Reinforcement Learning in Finance: Modeling Market Impact with Elliptic Uncertainty Sets [57.179679246370114]
  In financial applications, reinforcement learning (RL) agents are commonly trained on historical data, where their actions do not influence prices.<n>During deployment, these agents trade in live markets where their own transactions can shift asset prices, a phenomenon known as market impact.<n>Traditional robust RL approaches address this model misspecification by optimizing the worst-case performance over a set of uncertainties.<n>We develop a novel class of elliptic uncertainty sets, enabling efficient and tractable robust policy evaluation.
  arXiv  Detail & Related papers  (2025-10-22T18:22:25Z)
• Robust Offline Reinforcement Learning with Linearly Structured f-Divergence Regularization [11.739526562075339]
  The Robust Regularized Markov Decision Process (RRMDP) is proposed to learn policies robust to dynamics shifts by adding regularization to the transition dynamics in the value function.<n>We develop the Robust Regularized Pessimistic Value Iteration (R2PVI) algorithm that employs linear function approximation for robust policy learning in $d$-RRMDPs with $f$-divergence based regularization terms on transition kernels.
  arXiv  Detail & Related papers  (2024-11-27T18:57:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.