Automating Staged Rollout with Reinforcement Learning

• URL: http://arxiv.org/abs/2204.02189v1
• Date: Fri, 1 Apr 2022 21:22:39 GMT
• Title: Automating Staged Rollout with Reinforcement Learning
• Abstract summary: Staged rollout is a strategy of incrementally releasing software updates to portions of the user population in order to accelerate defect discovery without incurring catastrophic outcomes such as system wide outages. This paper demonstrates the potential to automate staged rollout with multi-objective reinforcement learning in order to dynamically balance stakeholder needs such as time to deliver new features and downtime incurred by failures due to latent defects.
• Score: 1.3750624267664155
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Staged rollout is a strategy of incrementally releasing software updates to portions of the user population in order to accelerate defect discovery without incurring catastrophic outcomes such as system wide outages. Some past studies have examined how to quantify and automate staged rollout, but stop short of simultaneously considering multiple product or process metrics explicitly. This paper demonstrates the potential to automate staged rollout with multi-objective reinforcement learning in order to dynamically balance stakeholder needs such as time to deliver new features and downtime incurred by failures due to latent defects.

Related papers

• Optimizing What Policies Learn From: Recoverability-aware Rollout Intervention Learning [21.808015281498076]
  Critic-free group-based reinforcement learning has become a scalable approach for post-training large language models.<n>Recent work has started to treat rollout generation as an adaptive decision, but two important limitations remain.<n>We propose Recoverability-Aware Learning (RAIL), a training-time framework that learns how to generate rollouts based on the improvement produced by each intervention.
  arXiv  Detail & Related papers  (2026-08-05T17:22:02Z)
• Unleashing the Potential of Large Language Models: A Blueprint for Real-Time, Enterprise-Ready Deployments [11.71713024399078]
  We present a unified, pattern-driven LLMOps architecture integrating real-time data ingestion, continual learning, retrieval-augmented generation, and human-in-the-loop feedback.<n>The result reduces latency-cost-accuracy trade-offs while supporting auditability and rollback for high-risk sectors such as health care and finance.
  arXiv  Detail & Related papers  (2026-08-01T03:40:22Z)
• Capturing Token Tendencies for Training-Free Token Pruning in Multimodal Large Language Models [86.02826269126308]
  We propose Trend-aware Pruning, a novel framework that elevates pruning from a local snapshot decision to a temporal trajectory modeling problem.<n>This enables a dynamic rectification mechanism that selectively reactivates "late-blooming" tokens, those initially undervalued but exhibiting rising semantic importance, thereby preventing the loss of critical visual cues.
  arXiv  Detail & Related papers  (2026-07-30T15:07:00Z)
• EfficientRollout: System-Aware Self-Speculative Decoding for RL Rollouts [20.460789503141466]
  We present EfficientRollout, a system-aware self-SD framework designed to address this gap for RL rollouts.<n> EfficientRollout reduces rollout and end-to-end latency by up to 19.6% and 12.7%, respectively, over an accelerated AR rollout baseline.
  arXiv  Detail & Related papers  (2026-06-17T11:51:06Z)
• Sequence of Expert: Boosting Imitation Planners for Autonomous Driving through Temporal Alternation [12.450883696383878]
  Imitation learning (IL) has emerged as a central paradigm in autonomous driving.<n>IL excels in matching expert behavior in open-loop settings by minimizing per-step prediction errors.<n>Over successive planning cycles, small, often imperceptible errors compound, potentially resulting in severe failures.<n>We propose Sequence of Experts (SoE) to enhance closed-loop performance without increasing model size or data requirements.
  arXiv  Detail & Related papers  (2025-12-15T08:50:23Z)
• Beat the long tail: Distribution-Aware Speculative Decoding for RL Training [75.75462952580796]
  We propose a Distribution Aware Speculative decoding framework that accelerates RL rollouts without altering model outputs.<n>Experiments on math and code reasoning tasks show that DAS reduces rollout time up to 50% while preserving identical training curves.
  arXiv  Detail & Related papers  (2025-11-17T19:02:12Z)
• Auto-Prompt Generation is Not Robust: Prompt Optimization Driven by Pseudo Gradient [50.15090865963094]
  We introduce PertBench, a comprehensive benchmark dataset that includes a wide range of input perturbations.<n>Our analysis reveals substantial vulnerabilities in existing prompt generation strategies.<n>We propose PGO, a gradient-free prompt generation framework that leverages perturbation types as pseudo-gradient signals.
  arXiv  Detail & Related papers  (2024-12-24T06:05:08Z)
• Code-as-Monitor: Constraint-aware Visual Programming for Reactive and Proactive Robotic Failure Detection [56.66677293607114]
  We propose Code-as-Monitor (CaM) for both open-set reactive and proactive failure detection. To enhance the accuracy and efficiency of monitoring, we introduce constraint elements that abstract constraint-related entities. Experiments show that CaM achieves a 28.7% higher success rate and reduces execution time by 31.8% under severe disturbances.
  arXiv  Detail & Related papers  (2024-12-05T18:58:27Z)
• Learning Quantitative Automata Modulo Theories [17.33092604696224]
  We present QUINTIC, an active learning algorithm, wherein the learner infers a valid automaton through deductive reasoning. Our evaluations utilize theory of rationals in order to learn summation, discounted summation, product, and classification quantitative automata.
  arXiv  Detail & Related papers  (2024-11-15T21:51:14Z)
• CORT: Class-Oriented Real-time Tracking for Embedded Systems [46.3107850275261]
  This work proposes a new approach to multi-class object tracking. It allows achieving smaller and more predictable execution times, without penalizing the tracking performance. The proposed solution was evaluated in complex urban scenarios with several objects of different types.
  arXiv  Detail & Related papers  (2024-07-20T09:12:17Z)
• Apparate: Rethinking Early Exits to Tame Latency-Throughput Tensions in ML Serving [10.926767319124547]
  We present Apparate, a system that automatically applies and manages early exits in machine learning models. To cope with the time-varying overhead and accuracy challenges that EEs bring, Apparate repurposes exits to provide continual feedback. Apparate lowers median response latencies by 40.5--91.5% and 10.0--24.2% for diverse CV and NLP classification workloads.
  arXiv  Detail & Related papers  (2023-12-08T21:49:09Z)
• Self-Supervised Multi-Object Tracking For Autonomous Driving From Consistency Across Timescales [53.55369862746357]
  Self-supervised multi-object trackers have tremendous potential as they enable learning from raw domain-specific data. However, their re-identification accuracy still falls short compared to their supervised counterparts. We propose a training objective that enables self-supervised learning of re-identification features from multiple sequential frames.
  arXiv  Detail & Related papers  (2023-04-25T20:47:29Z)
• Unsupervised Automata Learning via Discrete Optimization [7.06671668667062]
  We propose a framework for learning a deterministic finite automaton (DFA) from a given multi-set of unlabeled words.<n>We show that this problem is computationally hard and develop three learning algorithms based on constraint optimization.<n>We introduce novel regularization schemes for our optimization problems that improve the overall interpretability of our DFAs.
  arXiv  Detail & Related papers  (2023-03-24T16:19:15Z)
• Self-Supervised Representation Learning from Temporal Ordering of Automated Driving Sequences [49.91741677556553]
  We propose TempO, a temporal ordering pretext task for pre-training region-level feature representations for perception tasks. We embed each frame by an unordered set of proposal feature vectors, a representation that is natural for object detection or tracking systems. Extensive evaluations on the BDD100K, nuImages, and MOT17 datasets show that our TempO pre-training approach outperforms single-frame self-supervised learning methods.
  arXiv  Detail & Related papers  (2023-02-17T18:18:27Z)
• A Learning Based Framework for Handling Uncertain Lead Times in Multi-Product Inventory Management [8.889304968879163]
  Most existing literature on supply chain and inventory management consider demand processes with zero or constant lead times. Motivated by the recently introduced delay-resolved deep Q-learning (DRDQN) algorithm, this paper develops a reinforcement learning based paradigm for handling uncertainty in lead times.
  arXiv  Detail & Related papers  (2022-03-02T05:50:04Z)
• Automated Machine Learning Techniques for Data Streams [91.3755431537592]
  This paper surveys the state-of-the-art open-source AutoML tools, applies them to data collected from streams, and measures how their performance changes over time. The results show that off-the-shelf AutoML tools can provide satisfactory results but in the presence of concept drift, detection or adaptation techniques have to be applied to maintain the predictive accuracy over time.
  arXiv  Detail & Related papers  (2021-06-14T11:42:46Z)
• Induction and Exploitation of Subgoal Automata for Reinforcement Learning [75.55324974788475]
  We present ISA, an approach for learning and exploiting subgoals in episodic reinforcement learning (RL) tasks. ISA interleaves reinforcement learning with the induction of a subgoal automaton, an automaton whose edges are labeled by the task's subgoals. A subgoal automaton also consists of two special states: a state indicating the successful completion of the task, and a state indicating that the task has finished without succeeding.
  arXiv  Detail & Related papers  (2020-09-08T16:42:55Z)
• Bayesian active learning for production, a systematic study and a reusable library [85.32971950095742]
  In this paper, we analyse the main drawbacks of current active learning techniques. We do a systematic study on the effects of the most common issues of real-world datasets on the deep active learning process. We derive two techniques that can speed up the active learning loop such as partial uncertainty sampling and larger query size.
  arXiv  Detail & Related papers  (2020-06-17T14:51:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.