FAST: A Framework for Aligned Sampling and Training in Parallel Reinforcement Learning for Autonomous Driving

• URL: http://arxiv.org/abs/2606.21587v1
• Date: Fri, 19 Jun 2026 16:44:18 GMT
• Title: FAST: A Framework for Aligned Sampling and Training in Parallel Reinforcement Learning for Autonomous Driving
• Abstract summary: We propose FAST, a synchronous parallel framework tailored for closed-loop simulation.<n>FAST employs Dynamic Parallel Sampling Alignment (DPSA) to maintain vectorization synchronization.<n>FAST achieves at least a 1.78 times wall-clock speedup over the single-clip baseline.
• Score: 27.76982216294607
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Deep reinforcement learning is pivotal for closed-loop autonomous driving yet remains constrained by severe bottlenecks in sampling efficiency. Standard parallel sampling mitigates this but suffers from the straggler effect, where the premature termination of a single environment necessitates a synchronized batch re-initialization, leading to suboptimal sample utilization and prohibitive re-initialization latency. To address this, we propose FAST, a synchronous parallel framework tailored for closed-loop simulation. Specifically, FAST employs Dynamic Parallel Sampling Alignment (DPSA) to maintain vectorization synchronization by extending terminated episodes via virtual continuation, thereby decoupling the sampling loop from individual terminations. By dynamically triggering global truncation based on the termination rate of parallel clips, FAST effectively eliminates the bottleneck of premature resets without sacrificing data diversity. Furthermore, to strictly preserve theoretical consistency, we incorporate a Scaled Mask-Padding Optimization (SMPO) that leverages validity masking and adaptive loss normalization to nullify the bias from auxiliary padding data. Empirical evaluations demonstrate that FAST achieves at least a 1.78 times wall-clock speedup over the single-clip baseline while preserving statistical unbiasedness.

Related papers

• Attention-Discounted Adaptive Sampler for Masked Diffusion Language Models [59.51249894128724]
  Masked diffusion language models can reduce inference steps by revealing multiple tokens per denoising iteration.<n>We propose ADAS, a training-free reranking rule for parallel masked diffusion decoding.
  arXiv  Detail & Related papers  (2026-06-09T13:17:27Z)
• Breaking the Bubble: Asynchronous Pipeline Parallel Training with Bounded Weight Inconsistency [5.093144724934585]
  PACI is a bubble-free pipeline method that bounds forward/backward version drift without weight stashing, prediction, additional parameter copies, or global synchronization.<n>In GPT-style language-model pretraining, PACI matches the stability and final perplexity of synchronous 1F1B-flush, retains the same peak memory footprint, and achieves fully utilized pipeline throughput.<n>These results show that forward/backward inconsistency need not be eliminated: when explicitly bounded, it can be safely traded for substantial efficiency gains.
  arXiv  Detail & Related papers  (2026-06-05T22:33:57Z)
• Bringing Order to Asynchronous SGD: Towards Optimality under Data-Dependent Delays with Momentum [16.032739611868685]
  Asynchronous distributed gradient training (SGD) enables scalable distributed training but suffers from staleness.<n>Existing mitigation strategies, such as delay-dependent learning rates and staleness-aware training, typically or discard delayed gradients.<n>We propose a momentum-based asynchronous framework designed to preserve information from delayed gradients while mitigating staleness.
  arXiv  Detail & Related papers  (2026-05-03T20:24:24Z)
• CurvZO: Adaptive Curvature-Guided Sparse Zeroth-Order Optimization for Efficient LLM Fine-Tuning [15.930478833291827]
  Fine-tuning large language models (LLMs) with backpropagation achieves high performance but incurs substantial memory overhead.<n>We propose CurvZO, which tracks curvature signals online from scalar ZO feedback.<n>We show that CurvZO consistently improves fine-tuning performance and reduces training time over ZO baselines.
  arXiv  Detail & Related papers  (2026-03-23T09:13:45Z)
• Mitigating Staleness in Asynchronous Pipeline Parallelism via Basis Rotation [6.710608163117798]
  Asynchronous pipeline parallelism maximizes hardware utilization by eliminating the pipeline bubbles inherent in synchronous execution.<n>This efficiency gain can be compromised by gradient staleness, where the immediate model updates with delayed gradients introduce noise into the optimization process.<n>We propose the use of basis rotation, demonstrating that it effectively mitigates the alignment issue and significantly accelerates convergence in asynchronous settings.
  arXiv  Detail & Related papers  (2026-02-03T13:31:51Z)
• Edit-Based Flow Matching for Temporal Point Processes [51.33476564706644]
  temporal point processes (TPPs) are a fundamental tool for modeling event sequences in continuous time.<n>Recent non-autoregressive, diffusion-style models mitigate these issues by jointly interpolating between noise and data.<n>We introduce an Edit Flow process for TPPs that transports noise to data via insert, delete, and substitute edit operations.
  arXiv  Detail & Related papers  (2025-10-07T15:44:12Z)
• Ringleader ASGD: The First Asynchronous SGD with Optimal Time Complexity under Data Heterogeneity [51.56484100374058]
  We introduce Ringleader ASGD, the first asynchronous algorithm that attains the theoretical lower bounds for parallel computation.<n>Our analysis further establishes that Ringleader ASGD remains optimal under arbitrary gradient and even time-varying speeds.
  arXiv  Detail & Related papers  (2025-09-26T19:19:15Z)
• ATTS: Asynchronous Test-Time Scaling via Conformal Prediction [112.54016379556073]
  Large language models (LLMs) benefit from test-time scaling but are often hampered by high inference latency.<n>We introduce ATTS (Asynchronous Test-Time Scaling), a statistically guaranteed adaptive scaling framework.<n>We show that ATTS delivers up to 56.7x speedup in test-time scaling and a 4.14x throughput improvement.
  arXiv  Detail & Related papers  (2025-09-18T16:55:09Z)
• Adaptive Deadline and Batch Layered Synchronized Federated Learning [66.93447103966439]
  Federated learning (FL) enables collaborative model training across distributed edge devices while preserving data privacy, and typically operates in a round-based synchronous manner.<n>We propose ADEL-FL, a novel framework that jointly optimize per-round deadlines and user-specific batch sizes for layer-wise aggregation.
  arXiv  Detail & Related papers  (2025-05-29T19:59:18Z)
• Nesterov Method for Asynchronous Pipeline Parallel Optimization [59.79227116582264]
  We introduce a variant of Nesterov Accelerated Gradient (NAG) for asynchronous optimization in Pipeline Parallelism.<n>Specifically, we modify the look-ahead step in NAG to effectively address the staleness in gradients.<n>We theoretically prove that our approach converges at a sublinear rate in the presence of fixed delay in gradients.
  arXiv  Detail & Related papers  (2025-05-02T08:23:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.