Comparative Analysis of Parameterized Action Actor-Critic Reinforcement Learning Algorithms for Web Search Match Plan Generation

• URL: http://arxiv.org/abs/2510.03064v1
• Date: Fri, 03 Oct 2025 14:48:57 GMT
• Title: Comparative Analysis of Parameterized Action Actor-Critic Reinforcement Learning Algorithms for Web Search Match Plan Generation
• Abstract summary: This study evaluates the performance of Soft Actor Critic (SAC), Greedy Actor Critic (GAC), and Truncated Quantile Critics (TQC) in high-dimensional decision-making tasks.<n>The focus is on parametrized action (PA) spaces, eliminating the need for recurrent networks.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This study evaluates the performance of Soft Actor Critic (SAC), Greedy Actor Critic (GAC), and Truncated Quantile Critics (TQC) in high-dimensional decision-making tasks using fully observable environments. The focus is on parametrized action (PA) spaces, eliminating the need for recurrent networks, with benchmarks Platform-v0 and Goal-v0 testing discrete actions linked to continuous action-parameter spaces. Hyperparameter optimization was performed with Microsoft NNI, ensuring reproducibility by modifying the codebase for GAC and TQC. Results show that Parameterized Action Greedy Actor-Critic (PAGAC) outperformed other algorithms, achieving the fastest training times and highest returns across benchmarks, completing 5,000 episodes in 41:24 for the Platform game and 24:04 for the Robot Soccer Goal game. Its speed and stability provide clear advantages in complex action spaces. Compared to PASAC and PATQC, PAGAC demonstrated superior efficiency and reliability, making it ideal for tasks requiring rapid convergence and robust performance. Future work could explore hybrid strategies combining entropy-regularization with truncation-based methods to enhance stability and expand investigations into generalizability.

Related papers

• CAST: Critique-Aware Supervision for Training Reliable Long-Horizon Tool-Calling Agents [66.9575676266637]
  Large language model (LLM) agents are increasingly deployed in long-horizon, interactive, and stateful environments.<n>In these settings, a single wrong action, such as refunding the wrong purchase, can cause irreversible task failure and must be intercepted before execution.<n>We develop CAST, a critique-aware training framework that converts sparse task outcomes into action-level supervision for critique learning and policy optimization.
  arXiv  Detail & Related papers  (2026-08-31T01:59:51Z)
• Comparative Study of Multi-Agent Actor-Critic Algorithms in Parameterized Action Reinforcement Learning [0.0]
  Multi-Agent Greedy Actor-Critic (MAGAC), Multi-Agent Soft Actor-Critic (MASAC), and Multi-Agent Truncated Quantile Critics (MATQC)<n>We evaluate the algorithms on the Platform-v0 and Goal-v0 benchmarks against their single-agent counterparts.<n>Results show that the multi-agent framework consistently improves Greedy Actor-Critic performance.
  arXiv  Detail & Related papers  (2026-07-21T14:03:29Z)
• AMBUSH: Collaborative Capture in Complex Environments with Neural Acceleration [8.711548894933875]
  Collaborative capture of dynamic targets is common in nature as an essential strategy for weaker species against the strong.<n>This work tackles the problem from a unique perspective: the renowned strategy ofambush'' alone would suffice for multiple slower pursuers to capture one faster evader.<n>A Hybrid Monte Carlo Tree Search (H-MCTS) algorithm is proposed to optimize the associated parameters through long-term planning.<n>The neural acceleration is trained offline to learn the ranking of different choices of parameters across various environments, and to directly predict scores.
  arXiv  Detail & Related papers  (2026-07-01T14:59:28Z)
• DynaSchedBench: Calibrated Dynamic Scheduling Benchmarks and Observability Paradox in LLM-based Scheduling Agents [11.212756680882444]
  We introduce textbfDynaSchedBench, a diagnostic framework for DFJSP that rigorously controls the instance-generation process.<n>Instead of relying on parameter sampling, our approach utilizes Sequential Event-Space Calibrator (SESC) that computes a novel Schedule Stress Index (SSI) to stratify instances by difficulty.
  arXiv  Detail & Related papers  (2026-05-26T18:36:54Z)
• Actor-Critic with Active Importance Sampling [61.41272490044518]
  This paper introduces the Active-Importance-Sampling Actor-Critic (AISAC) algorithm.<n>AISAC is an extension of the Actor-Critic framework for reducing variance in policy gradient estimation.<n> Experiments show improved learning speed, sample efficiency, and training compared to standard Actor-Critic methods.
  arXiv  Detail & Related papers  (2026-05-08T01:21:32Z)
• CTS-PLL: A Robust and Anytime Framework for Collaborative Task Sequencing and Multi-Agent Path Finding [11.988773389912573]
  The Collaborative Task Sequencing and Multi-Agent Path Finding (CTS-MAPF) problem requires agents to accomplish sequences of tasks while avoiding collisions.<n>This work introduces CTS-PLL, a hierarchical framework that extends the configuration-based CTS-MAPF planning paradigm with two key enhancements.
  arXiv  Detail & Related papers  (2026-03-26T07:45:44Z)
• TTF-VLA: Temporal Token Fusion via Pixel-Attention Integration for Vision-Language-Action Models [29.878993349922368]
  Vision-Language-Action (VLA) models process visual inputs independently at each timestep, discarding valuable temporal information inherent in robotic manipulation tasks.<n>We propose Temporal Token Fusion (TTF), a training-free approach that integrates historical and current visual representations to enhance VLA inference quality.
  arXiv  Detail & Related papers  (2025-08-15T12:03:34Z)
• Fast Adaptation with Behavioral Foundation Models [82.34700481726951]
  Unsupervised zero-shot reinforcement learning has emerged as a powerful paradigm for pretraining behavioral foundation models.<n>Despite promising results, zero-shot policies are often suboptimal due to errors induced by the unsupervised training process.<n>We propose fast adaptation strategies that search in the low-dimensional task-embedding space of the pre-trained BFM to rapidly improve the performance of its zero-shot policies.
  arXiv  Detail & Related papers  (2025-04-10T16:14:17Z)
• Performance-driven Constrained Optimal Auto-Tuner for MPC [36.143463447995536]
  We propose COAT-MPC, Constrained Optimal Auto-Tuner for MPC.<n>COAT-MPC gathers performance data and learns by updating its posterior belief.<n>We theoretically analyze COAT-MPC, showing that it satisfies performance constraints with arbitrarily high probability.
  arXiv  Detail & Related papers  (2025-03-10T09:56:08Z)
• SETS: Leveraging Self-Verification and Self-Correction for Improved Test-Time Scaling [39.57154199908565]
  Self-Enhanced Test-Time Scaling (SETS) is a simple yet effective approach that overcomes limitations by strategically combining parallel and sequential techniques.<n>SETS exploits the inherent self-verification and self- computation capabilities of Large Language Models, unifying sampling, verification, and correction within a single framework.<n>Our results show SETS achieves significant performance improvements and more advantageous test-time scaling behavior than the alternatives.
  arXiv  Detail & Related papers  (2025-01-31T17:03:16Z)
• Faster Diffusion Action Segmentation [9.868244939496678]
  Temporal Action Classification (TAS) is an essential task in video analysis, aiming to segment and classify continuous frames into distinct action segments. Recent advances in diffusion models have demonstrated substantial success in TAS tasks due to their stable training process and high-quality generation capabilities. We propose EffiDiffAct, an efficient and high-performance TAS algorithm.
  arXiv  Detail & Related papers  (2024-08-04T13:23:18Z)
• Hyperparameters in Continual Learning: A Reality Check [53.30082523545212]
  Continual learning (CL) aims to train a model on a sequence of tasks while balancing the trade-off between plasticity (learning new tasks) and stability (retaining prior knowledge)
  arXiv  Detail & Related papers  (2024-03-14T03:13:01Z)
• Maximum Entropy Heterogeneous-Agent Reinforcement Learning [45.377385280485065]
  Multi-agent reinforcement learning (MARL) has been shown effective for cooperative games in recent years.<n>We propose a unified framework for learning policies to resolve issues related to sample complexity, training instability, and the risk of converging to a suboptimal Nash Equilibrium.<n>Based on the MaxEnt framework, we propose Heterogeneous-Agent Soft Actor-Critic (HASAC) algorithm.<n>We evaluate HASAC on six benchmarks: Bi-DexHands, Multi-Agent MuJoCo, StarCraft Challenge, Google Research Football, Multi-Agent Particle Environment, and Light Aircraft Game.
  arXiv  Detail & Related papers  (2023-06-19T06:22:02Z)
• MARLIN: Soft Actor-Critic based Reinforcement Learning for Congestion Control in Real Networks [63.24965775030673]
  We propose a novel Reinforcement Learning (RL) approach to design generic Congestion Control (CC) algorithms. Our solution, MARLIN, uses the Soft Actor-Critic algorithm to maximize both entropy and return. We trained MARLIN on a real network with varying background traffic patterns to overcome the sim-to-real mismatch.
  arXiv  Detail & Related papers  (2023-02-02T18:27:20Z)
• Solving Continuous Control via Q-learning [54.05120662838286]
  We show that a simple modification of deep Q-learning largely alleviates issues with actor-critic methods. By combining bang-bang action discretization with value decomposition, framing single-agent control as cooperative multi-agent reinforcement learning (MARL), this simple critic-only approach matches performance of state-of-the-art continuous actor-critic methods.
  arXiv  Detail & Related papers  (2022-10-22T22:55:50Z)
• Dealing with Sparse Rewards in Continuous Control Robotics via Heavy-Tailed Policies [64.2210390071609]
  We present a novel Heavy-Tailed Policy Gradient (HT-PSG) algorithm to deal with the challenges of sparse rewards in continuous control problems. We show consistent performance improvement across all tasks in terms of high average cumulative reward.
  arXiv  Detail & Related papers  (2022-06-12T04:09:39Z)
• Evolving Pareto-Optimal Actor-Critic Algorithms for Generalizability and Stability [67.8426046908398]
  Generalizability and stability are two key objectives for operating reinforcement learning (RL) agents in the real world. This paper presents MetaPG, an evolutionary method for automated design of actor-critic loss functions.
  arXiv  Detail & Related papers  (2022-04-08T20:46:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.