Target-Aligned Reinforcement Learning

• URL: http://arxiv.org/abs/2603.29501v1
• Date: Tue, 31 Mar 2026 09:42:37 GMT
• Title: Target-Aligned Reinforcement Learning
• Abstract summary: Many reinforcement learning algorithms rely on target networks - lagged copies of the online network - to stabilize training.<n>We propose Target-Aligned Reinforcement Learning (TARL), a framework that emphasizes transitions for which the target and online network estimates are highly aligned.
• Score: 12.477207334328156
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Many reinforcement learning algorithms rely on target networks - lagged copies of the online network - to stabilize training. While effective, this mechanism introduces a fundamental stability-recency tradeoff: slower target updates improve stability but reduce the recency of learning signals, hindering convergence speed. We propose Target-Aligned Reinforcement Learning (TARL), a framework that emphasizes transitions for which the target and online network estimates are highly aligned. By focusing updates on well-aligned targets, TARL mitigates the adverse effects of stale target estimates while retaining the stabilizing benefits of target networks. We provide a theoretical analysis demonstrating that target alignment correction accelerates convergence, and empirically demonstrate consistent improvements over standard reinforcement learning algorithms across various benchmark environments.

Related papers

• Decentralized Learning Strategies for Estimation Error Minimization with Graph Neural Networks [86.99017195607077]
  We address real-time sampling and estimation of autoregressive Markovian sources in wireless networks.<n>We propose a graphical reinforcement learning framework for policy optimization.<n>Theoretically, our proposed policies are transferable, allowing a policy trained on one graph to be effectively applied to structurally similar graphs.
  arXiv  Detail & Related papers  (2026-01-19T02:18:45Z)
• Resolving Conflicts in Lifelong Learning via Aligning Updates in Subspaces [12.630494786258842]
  Low-Rank Adaptation (LoRA) enables efficient Continual Learning but often suffers from catastrophic forgetting.<n>We propose PS-LoRA, a framework designed to resolve conflicts by aligning updates within the optimization subspace.<n>Our approach employs a dual-regularization objective that penalizes conflicting directions and constrains magnitude deviations to ensure consistency with prior knowledge.
  arXiv  Detail & Related papers  (2025-11-28T15:34:36Z)
• Use the Online Network If You Can: Towards Fast and Stable Reinforcement Learning [22.6796319984868]
  We introduce a novel update rule that computes the target using the MINimum estimate between the Target and Online network.<n> MINTO enables faster and stable value function learning, by mitigating the potential overestimation bias of using the online network for bootstrapping.<n>We evaluate MINTO extensively across diverse benchmarks, spanning online and offline RL, as well as discrete and continuous action spaces.
  arXiv  Detail & Related papers  (2025-10-02T21:48:01Z)
• Network Sparsity Unlocks the Scaling Potential of Deep Reinforcement Learning [57.3885832382455]
  We show that introducing static network sparsity alone can unlock further scaling potential beyond dense counterparts with state-of-the-art architectures.<n>Our analysis reveals that, in contrast to naively scaling up dense DRL networks, such sparse networks achieve both higher parameter efficiency for network expressivity.
  arXiv  Detail & Related papers  (2025-06-20T17:54:24Z)
• Bridging the Performance Gap Between Target-Free and Target-Based Reinforcement Learning [21.38951369323128]
  We introduce a new method that uses a copy of the last linear layer of the online network as a target network.<n>We find that combining our approach with the concept of iterated Q-learning, which consists of learning consecutive Bellman updates in parallel, helps improve the sample-efficiency of target-free approaches.
  arXiv  Detail & Related papers  (2025-06-04T19:27:29Z)
• Learn from the Past: A Proxy Guided Adversarial Defense Framework with Self Distillation Regularization [53.04697800214848]
  Adversarial Training (AT) is pivotal in fortifying the robustness of deep learning models. AT methods, relying on direct iterative updates for target model's defense, frequently encounter obstacles such as unstable training and catastrophic overfitting. We present a general proxy guided defense framework, LAST' (bf Learn from the Pbf ast)
  arXiv  Detail & Related papers  (2023-10-19T13:13:41Z)
• Doubly Robust Instance-Reweighted Adversarial Training [107.40683655362285]
  We propose a novel doubly-robust instance reweighted adversarial framework. Our importance weights are obtained by optimizing the KL-divergence regularized loss function. Our proposed approach outperforms related state-of-the-art baseline methods in terms of average robust performance.
  arXiv  Detail & Related papers  (2023-08-01T06:16:18Z)
• Why Target Networks Stabilise Temporal Difference Methods [38.35578010611503]
  We show that under mild regularity conditions and a well tuned target network update frequency, convergence can be guaranteed. We conclude that the use of target networks can mitigate the effects of poor conditioning in the Jacobian of the TD update.
  arXiv  Detail & Related papers  (2023-02-24T09:46:00Z)
• Bridging the Gap Between Target Networks and Functional Regularization [61.051716530459586]
  We propose an explicit Functional Regularization that is a convex regularizer in function space and can easily be tuned. We analyze the convergence of our method theoretically and empirically demonstrate that replacing Target Networks with the more theoretically grounded Functional Regularization approach leads to better sample efficiency and performance improvements.
  arXiv  Detail & Related papers  (2022-10-21T22:27:07Z)
• Bridging the Gap Between Target Networks and Functional Regularization [61.051716530459586]
  We show that Target Networks act as an implicit regularizer which can be beneficial in some cases, but also have disadvantages. We propose an explicit Functional Regularization alternative that is flexible and a convex regularizer in function space. Our findings emphasize that Functional Regularization can be used as a drop-in replacement for Target Networks and result in performance improvement.
  arXiv  Detail & Related papers  (2021-06-04T17:21:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.