Reinforcement Learning for Execution under Dynamic Fees in a Closed-Loop DEX Simulator

• URL: http://arxiv.org/abs/2607.10960v1
• Date: Sun, 12 Jul 2026 23:47:16 GMT
• Title: Reinforcement Learning for Execution under Dynamic Fees in a Closed-Loop DEX Simulator
• Abstract summary: Trader-facing dynamic fees are increasingly proposed for automated market makers (AMMs)<n>historical data do not identify how order flow would respond: trader-facing fees do not vary, trader types are latent, and a replayed tape is not a sequential decision environment.<n>We construct a minimal closed-loop simulator in which the missing signal exists by construction.
• Score: 0.8528384027684189
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Trader-facing dynamic fees are increasingly proposed for automated market makers (AMMs), but historical data do not identify how order flow would respond: trader-facing fees do not vary, trader types are latent, and a replayed tape is not a sequential decision environment. We therefore construct a minimal closed-loop simulator in which the missing signal exists by construction: two constant-product pools repriced by an equilibrium-inspired dynamic-fee rule, fee-sensitive noise flow, and closed-form CEX--AMM arbitrage. Equilibrium is used as a closure principle, not as an object the trader learns. Against a tuned benchmark ladder of schedule, planning, lookahead, and tabular policies, a small DQN is the only evaluated valid policy whose paired improvement over tuned one-step routing excludes zero. On a reserved final block of 1{,}000 seeds with completion forced to 1.0 for every policy, it reduces implementation shortfall under every tested intra-step ordering, by $13.3\bps$ of order notional under the pre-specified agent-last ordering, and the edge is concentrated in, and learned from, dynamic-fee environments: under constant fees the paired difference is indistinguishable from zero. The result is model-conditioned counterfactual evidence about execution control in AMMs, not evidence about historical traders, equilibrium play, or deployable profit.

Related papers

• CLQT: A Closed-Loop, Cost-Aware, Strategy-Consistent Benchmark for Diagnostic Evaluation of LLM Portfolio-Management Agents [0.9924185669370708]
  We introduce CLQT, which reframes closed-loop trading evaluation as diagnosis rather than ranking.<n> CLQT is a fully closed-loop, cost-aware, strategy-consistent, temporally-gated environment.
  arXiv  Detail & Related papers  (2026-06-29T04:26:30Z)
• TAPO: Tool-Aware Policy Optimization via Credit Transfer for Multimodal Search Agents [54.08846865906602]
  We characterize credit misassignment as a systematic failure mode of GRPO in tool-augmented multimodal search agents.<n>We propose Tool-Aware Policy Optimization (TAPO), which exploits the parameter-determinism property of information-acquisition tools.
  arXiv  Detail & Related papers  (2026-06-04T07:15:43Z)
• Scaling Laws for Agent Harnesses via Effective Feedback Compute [53.68149869349268]
  emphEffective Feedback Compute (EFC) is a trace-level scaling coordinate that credits feedback only when it is informative, valid, non-redundant, and retained for subsequent decisions.<n>EFC-based coordinates consistently predict failure rates better than raw-compute baselines.
  arXiv  Detail & Related papers  (2026-05-28T09:45:47Z)
• BAPR: Bayesian amnesic piecewise-robust reinforcement learning for non-stationary continuous control [19.34848029403215]
  Real-world control systems operate under emphpiecewise stationary conditions, where dynamics remain stable for extended periods before undergoing regime changes.<n>Standard robust RL methods face a fundamental dilemma: a globally conservative policy wastes performance during stable periods, while a locally adaptive policy risks catastrophic failure when the regime changes undetected.<n>We propose textbfBAPR, which unifies Bayesian Online Change Detection with robust ensemble RL.
  arXiv  Detail & Related papers  (2026-05-15T16:49:05Z)
• Synthetic American Option Pricing via Jump-HMM-Driven Heston Implied Volatility [5.087681397628885]
  We develop a synthetic-data generator for synthetic option prices based on implied volatility.<n>We apply the framework as a synthetic-data generator on real near-the-money put and call contracts, forward-simulating price paths, and recovering path-conditional implied volatility.<n>The framework is released as an open-source Julia package.
  arXiv  Detail & Related papers  (2026-05-13T18:07:14Z)
• ValueBlindBench: Agreement-Gated Stress Testing of LLM-Judged Investment Rationales Before Returns Are Observable [1.8689252029357564]
  This paper introduces ValueBlindBench, a preregistered agreement-gated stress-test protocol.<n>ValueBlindBench decides when LLM-judged investment-rationale claims are publishable, qualified, or invalid.
  arXiv  Detail & Related papers  (2026-04-28T05:04:20Z)
• Anchor-and-Resume Concession Under Dynamic Pricing for LLM-Augmented Freight Negotiation [5.225967186005105]
  Deriving $$ from the live spread enables adaptation but introduces a new problem: a pricing shift can cause the formula to retract a previous offer, violating monotonicity.<n>We propose a two-index anchor-and-resume framework that addresses both limitations.<n>A spread-derived $$ maps each load's margin structure to the correct concession posture, while the anchor-and-resume mechanism guarantees monotonically nondecreasing offers under arbitrary pricing shifts.
  arXiv  Detail & Related papers  (2026-04-22T16:17:07Z)
• Unleashing Implicit Rewards: Prefix-Value Learning for Distribution-Level Optimization [74.91418266859297]
  Implicit process reward models (PRMs) provide fine-grained reward signals along the reasoning process.<n>PRMs suffer from a train-inference mismatch: training only constrains a sequence-level aggregate, whereas inference requires token-level scores to reflect local step quality.<n>We propose a novel Implicit Prefix-Value Reward Model (IPVRM), which directly learns a prefix-conditioned value function estimating the probability of eventual correctness.<n>We also propose Distribution-Level RL (DistRL), which computes TD advantages for both sampled tokens and high-probability candidate tokens.
  arXiv  Detail & Related papers  (2026-04-14T18:19:54Z)
• Improving Search Agent with One Line of Code [68.58667107354253]
  Tool-based Agentic Reinforcement Learning (TARL) has emerged as a promising paradigm for training search agents to interact with external tools.<n>We propose textbfSearch textbfAgent textbfPolicy textbfOptimization (textbfSAPO), which stabilizes training via a conditional token-level KL constraint.
  arXiv  Detail & Related papers  (2026-03-10T04:07:39Z)
• Unifying Stable Optimization and Reference Regularization in RLHF [64.16830602324345]
  This paper introduces a unified regularization approach that balances objectives of preventing reward hacking and maintaining stable policy updates.<n>Our simple yet principled alignment objective yields a weighted supervised fine-tuning loss with a superior trade-off, which demonstrably improves both alignment results and implementation complexity.
  arXiv  Detail & Related papers  (2026-02-12T03:31:19Z)
• Robust Offline Reinforcement Learning for Non-Markovian Decision Processes [48.9399496805422]
  We study the learning problem of robust offline non-Markovian RL.<n>We introduce a novel dataset distillation and a lower confidence bound (LCB) design for robust values under different types of the uncertainty set.<n>By further introducing a novel type-I concentrability coefficient tailored for offline low-rank non-Markovian decision processes, we prove that our algorithm can find an $epsilon$-optimal robust policy.
  arXiv  Detail & Related papers  (2024-11-12T03:22:56Z)
• Autoregressive Bandits [58.46584210388307]
  We propose a novel online learning setting, Autoregressive Bandits, in which the observed reward is governed by an autoregressive process of order $k$. We show that, under mild assumptions on the reward process, the optimal policy can be conveniently computed. We then devise a new optimistic regret minimization algorithm, namely, AutoRegressive Upper Confidence Bound (AR-UCB), that suffers sublinear regret of order $widetildemathcalO left( frac(k+1)3/2sqrtnT (1-G
  arXiv  Detail & Related papers  (2022-12-12T21:37:36Z)
• Bandit Linear Optimization for Sequential Decision Making and Extensive-Form Games [102.23975166536326]
  Tree-form sequential decision making (TFSDM) extends classical one-shot decision making by modeling tree-form interactions between an agent and a potentially adversarial environment. It captures the online decision-making problems that each player faces in an extensive-form game, as well as Markov decision processes and partially-observable Markov decision processes where the agent conditions on observed history. In this paper, we give the first algorithm for the bandit linear optimization problem for dilatedDM that offers both (i) linear-time losses and (ii) $O(sqrtT)$ cumulative regret in
  arXiv  Detail & Related papers  (2021-03-08T05:00:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.