Lighthouse RL: Sample-Efficient Circuit Optimization via Strategic Reset Points

• URL: http://arxiv.org/abs/2607.14008v1
• Date: Wed, 15 Jul 2026 16:37:57 GMT
• Title: Lighthouse RL: Sample-Efficient Circuit Optimization via Strategic Reset Points
• Abstract summary: We introduce Lighthouse, a sample-efficient reinforcement learning (RL) approach for analog circuit sizing.<n>We demonstrate the effectiveness of our approach on a 2D benchmark problem and on two analog circuits.<n>Our reset strategy can be used as a plug-and-play enhancement for any RL-based optimization approach.
• Score: 5.713714247838241
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this paper, we introduce Lighthouse RL, a sample-efficient reinforcement learning (RL) approach for analog circuit sizing. Traditional methods lack generalization across different performance targets, while standard RL approaches waste resources exploring unpromising regions. Our method addresses these inefficiencies through a strategic reset strategy that initializes episodes from high-performing configurations discovered during training, called "lighthouses". These states, which are closer to the target objectives, guide exploration toward promising regions. When compared to RL and Bayesian optimization methods from the literature, we demonstrate the effectiveness of our approach on a 2D benchmark problem and on two analog circuits, showing significant improvements in sample efficiency (up to 1.72x faster), optimization performance (100% vs. 0-87% success rate), generalization (75% vs. 0-50% extrapolation success), and objective maximization. This efficiency is particularly valuable for computationally expensive black-box optimization problems, and our reset strategy can be used as a plug-and-play enhancement for any RL-based optimization approach.

Related papers

• Nudging Beyond the Comfort Zone: Efficient Strategy-Guided Exploration for RLVR [53.27792011950384]
  Reinforcement learning with verifiable rewards (RLVR) has emerged as a scalable paradigm for improving the reasoning capabilities of large language models.<n>We propose NudgeRL, a framework for structured and diversity-driven exploration in RLVR.<n>Our approach introduces Strategy Nudging, which conditions each rollout on lightweight, strategy-level contexts.
  arXiv  Detail & Related papers  (2026-05-15T08:22:59Z)
• Evolution of Optimization Methods: Algorithms, Scenarios, and Evaluations [98.44542103979735]
  First-order gradient descent methods, epitomized by gradient descent (SGD) and Adam, serve as the cornerstone of modern training pipelines.<n>Large-scale model training, stringent privacy requirements, and distributed learning paradigms expose critical limitations in these conventional approaches regarding privacy protection and memory efficiency.<n>We retrospectively analyze the evolutionary trajectory of deep learning optimization algorithms and present a comprehensive empirical evaluation of mainstreams across diverse model architectures and training scenarios.<n>We distill key emerging trends and fundamental design trade-offs, pinpointing promising directions for future research.
  arXiv  Detail & Related papers  (2026-04-14T17:01:36Z)
• A New Trajectory-Oriented Approach to Enhancing Comprehensive Crowd Navigation Performance [28.07757592239467]
  Crowd navigation has garnered considerable research interest in recent years.<n>Current DRL approaches generally prioritize efficiency and comfort.<n>We propose a novel reward-shaping strategy that explicitly emphasizes trajectory-curvature optimization.
  arXiv  Detail & Related papers  (2025-12-07T00:52:07Z)
• Sample-efficient LLM Optimization with Reset Replay [13.739451157239756]
  We introduce Reset Replay (LoRR), a plugin designed to enhance sample efficiency in any preference-based optimization framework.<n>LoRR incorporates a periodic reset strategy with reusing initial data, which preserves network plasticity.<n>Our experiments demonstrate that LoRR significantly boosts the performance of various preference optimization methods on both mathematical and general reasoning benchmarks.
  arXiv  Detail & Related papers  (2025-08-08T15:56:49Z)
• Accelerating RL for LLM Reasoning with Optimal Advantage Regression [52.0792918455501]
  We propose a novel two-stage policy optimization framework that directly approximates the optimal advantage function.<n>$A$*-PO achieves competitive performance across a wide range of mathematical reasoning benchmarks.<n>It reduces training time by up to 2$times$ and peak memory usage by over 30% compared to PPO, GRPO, and REBEL.
  arXiv  Detail & Related papers  (2025-05-27T03:58:50Z)
• Preference Optimization for Combinatorial Optimization Problems [54.87466279363487]
  Reinforcement Learning (RL) has emerged as a powerful tool for neural optimization, enabling models learns that solve complex problems without requiring expert knowledge.<n>Despite significant progress, existing RL approaches face challenges such as diminishing reward signals and inefficient exploration in vast action spaces.<n>We propose Preference Optimization, a novel method that transforms quantitative reward signals into qualitative preference signals via statistical comparison modeling.
  arXiv  Detail & Related papers  (2025-05-13T16:47:00Z)
• Scattered Forest Search: Smarter Code Space Exploration with LLMs [55.71665969800222]
  We propose SCATTERED FOREST SEARCH (SFS), a novel approach that improves solution diversity and better exploits feedback during evolutionary search.<n>Our approach scales more efficiently than existing search techniques, including tree search, line search, and repeated sampling.
  arXiv  Detail & Related papers  (2024-10-22T01:58:29Z)
• Analyzing and Bridging the Gap between Maximizing Total Reward and Discounted Reward in Deep Reinforcement Learning [17.245293915129942]
  The optimal objective is a fundamental aspect of reinforcement learning (RL)<n>While total return is ideal, discounted return is practical objective due to its stability.<n>We propose two alternative approaches to align the objectives.
  arXiv  Detail & Related papers  (2024-07-18T08:33:10Z)
• Optimal Goal-Reaching Reinforcement Learning via Quasimetric Learning [73.80728148866906]
  Quasimetric Reinforcement Learning (QRL) is a new RL method that utilizes quasimetric models to learn optimal value functions. On offline and online goal-reaching benchmarks, QRL also demonstrates improved sample efficiency and performance.
  arXiv  Detail & Related papers  (2023-04-03T17:59:58Z)
• Deep Black-Box Reinforcement Learning with Movement Primitives [15.184283143878488]
  We present a new algorithm for deep reinforcement learning (RL) It is based on differentiable trust region layers, a successful on-policy deep RL algorithm. We compare our ERL algorithm to state-of-the-art step-based algorithms in many complex simulated robotic control tasks.
  arXiv  Detail & Related papers  (2022-10-18T06:34:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.