Enhancing Reinforcement Learning for the Floorplanning of Analog ICs with Beam Search

• URL: http://arxiv.org/abs/2505.05059v1
• Date: Thu, 08 May 2025 08:50:32 GMT
• Title: Enhancing Reinforcement Learning for the Floorplanning of Analog ICs with Beam Search
• Authors: Sandro Junior Della Rovere, Davide Basso, Luca Bortolussi, Mirjana Videnovic-Misic, Husni Habal,
• Abstract summary: This paper presents a hybrid method that combines reinforcement learning (RL) with a beam (BS) strategy.<n>The BS algorithm enhances the agent's inference process, allowing for the generation of flexible floorplans.<n> Experimental results show approx. 5-85% improvement in area, dead space and half-perimeter wire length compared to a standard RL application.
• Score: 0.32985979395737786
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: The layout of analog ICs requires making complex trade-offs, while addressing device physics and variability of the circuits. This makes full automation with learning-based solutions hard to achieve. However, reinforcement learning (RL) has recently reached significant results, particularly in solving the floorplanning problem. This paper presents a hybrid method that combines RL with a beam (BS) strategy. The BS algorithm enhances the agent's inference process, allowing for the generation of flexible floorplans by accomodating various objective weightings, and addressing congestion without without the need for policy retraining or fine-tuning. Moreover, the RL agent's generalization ability stays intact, along with its efficient handling of circuit features and constraints. Experimental results show approx. 5-85% improvement in area, dead space and half-perimeter wire length compared to a standard RL application, along with higher rewards for the agent. Moreover, performance and efficiency align closely with those of existing state-of-the-art techniques.

Related papers

• Ring-lite: Scalable Reasoning via C3PO-Stabilized Reinforcement Learning for LLMs [51.21041884010009]
  Ring-lite is a Mixture-of-Experts (MoE)-based large language model optimized via reinforcement learning (RL)<n>Our approach matches the performance of state-of-the-art (SOTA) small-scale reasoning models on challenging benchmarks.
  arXiv  Detail & Related papers  (2025-06-17T17:12:34Z)
• CHEQ-ing the Box: Safe Variable Impedance Learning for Robotic Polishing [5.467140383171385]
  This work presents an experimental demonstration of the hybrid RL algorithm CHEQ for robotic polishing with variable impedance.<n>On hardware, CHEQ achieves effective polishing behaviour, requiring only eight hours of training and incurring just five failures.<n>Results highlight the potential of adaptive hybrid RL for real-world, contact-rich tasks trained directly on hardware.
  arXiv  Detail & Related papers  (2025-01-14T10:13:41Z)
• Enhancing Spectrum Efficiency in 6G Satellite Networks: A GAIL-Powered Policy Learning via Asynchronous Federated Inverse Reinforcement Learning [67.95280175998792]
  A novel adversarial imitation learning (GAIL)-powered policy learning approach is proposed for optimizing beamforming, spectrum allocation, and remote user equipment (RUE) association ins. We employ inverse RL (IRL) to automatically learn reward functions without manual tuning. We show that the proposed MA-AL method outperforms traditional RL approaches, achieving a $14.6%$ improvement in convergence and reward value.
  arXiv  Detail & Related papers  (2024-09-27T13:05:02Z)
• PYRA: Parallel Yielding Re-Activation for Training-Inference Efficient Task Adaptation [61.57833648734164]
  We propose a novel Parallel Yielding Re-Activation (PYRA) method for training-inference efficient task adaptation. PYRA outperforms all competing methods under both low compression rate and high compression rate.
  arXiv  Detail & Related papers  (2024-03-14T09:06:49Z)
• Optimal Parallelization Strategies for Active Flow Control in Deep Reinforcement Learning-Based Computational Fluid Dynamics [29.49913315698914]
  Deep Reinforcement Learning (DRL) has emerged as a promising approach for handling highly dynamic and nonlinear Active Flow Control (AFC) problems. This study focuses on optimizing DRL-based algorithms in parallel settings. We achieve a significant boost in parallel efficiency from around 49% to approximately 78%.
  arXiv  Detail & Related papers  (2024-02-18T09:07:30Z)
• Transfer of Reinforcement Learning-Based Controllers from Model- to Hardware-in-the-Loop [1.8218298349840023]
  Reinforcement Learning has great potential for autonomously training agents to perform complex control tasks. To use RL effectively in embedded system function development, the generated agents must be able to handle real-world applications. This work focuses on accelerating the training process of RL agents by combining Transfer Learning (TL) and X-in-the-Loop (XiL) simulation.
  arXiv  Detail & Related papers  (2023-10-25T09:13:12Z)
• Hybrid Reinforcement Learning for Optimizing Pump Sustainability in Real-World Water Distribution Networks [55.591662978280894]
  This article addresses the pump-scheduling optimization problem to enhance real-time control of real-world water distribution networks (WDNs) Our primary objectives are to adhere to physical operational constraints while reducing energy consumption and operational costs. Traditional optimization techniques, such as evolution-based and genetic algorithms, often fall short due to their lack of convergence guarantees.
  arXiv  Detail & Related papers  (2023-10-13T21:26:16Z)
• Reinforcement Learning for Branch-and-Bound Optimisation using Retrospective Trajectories [72.15369769265398]
  Machine learning has emerged as a promising paradigm for branching. We propose retro branching; a simple yet effective approach to RL for branching. We outperform the current state-of-the-art RL branching algorithm by 3-5x and come within 20% of the best IL method's performance on MILPs with 500 constraints and 1000 variables.
  arXiv  Detail & Related papers  (2022-05-28T06:08:07Z)
• Reinforcement Learning-Empowered Mobile Edge Computing for 6G Edge Intelligence [76.96698721128406]
  Mobile edge computing (MEC) considered a novel paradigm for computation and delay-sensitive tasks in fifth generation (5G) networks and beyond. This paper provides a comprehensive research review on free-enabled RL and offers insight for development.
  arXiv  Detail & Related papers  (2022-01-27T10:02:54Z)
• Curriculum Based Reinforcement Learning of Grid Topology Controllers to Prevent Thermal Cascading [0.19116784879310028]
  This paper describes how domain knowledge of power system operators can be integrated into reinforcement learning frameworks. A curriculum-based approach with reward tuning is incorporated into the training procedure by modifying the environment. A parallel training approach on multiple scenarios is employed to avoid biasing the agent to a few scenarios and make it robust to the natural variability in grid operations.
  arXiv  Detail & Related papers  (2021-12-18T20:32:05Z)
• A Distributed Deep Reinforcement Learning Technique for Application Placement in Edge and Fog Computing Environments [31.326505188936746]
  Several Deep Reinforcement Learning (DRL)-based placement techniques have been proposed in fog/edge computing environments. We propose an actor-critic-based distributed application placement technique, working based on the IMPortance weighted Actor-Learner Architectures (IMPALA)
  arXiv  Detail & Related papers  (2021-10-24T11:25:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.