PICBench: Benchmarking LLMs for Photonic Integrated Circuits Design

• URL: http://arxiv.org/abs/2502.03159v1
• Date: Wed, 05 Feb 2025 13:32:29 GMT
• Title: PICBench: Benchmarking LLMs for Photonic Integrated Circuits Design
• Authors: Yuchao Wu, Xiaofei Yu, Hao Chen, Yang Luo, Yeyu Tong, Yuzhe Ma,
• Abstract summary: Large language models (LLMs) have shown remarkable potential in automating various tasks in digital chip design. We introduce PICBench, the first benchmarking and evaluation framework specifically designed to automate PIC design generation. Our benchmark consists of dozens of meticulously crafted PIC design problems, spanning from fundamental device designs to more complex circuit-level designs.
• Score: 11.843730779173491
• License:
• Abstract: While large language models (LLMs) have shown remarkable potential in automating various tasks in digital chip design, the field of Photonic Integrated Circuits (PICs)-a promising solution to advanced chip designs-remains relatively unexplored in this context. The design of PICs is time-consuming and prone to errors due to the extensive and repetitive nature of code involved in photonic chip design. In this paper, we introduce PICBench, the first benchmarking and evaluation framework specifically designed to automate PIC design generation using LLMs, where the generated output takes the form of a netlist. Our benchmark consists of dozens of meticulously crafted PIC design problems, spanning from fundamental device designs to more complex circuit-level designs. It automatically evaluates both the syntax and functionality of generated PIC designs by comparing simulation outputs with expert-written solutions, leveraging an open-source simulator. We evaluate a range of existing LLMs, while also conducting comparative tests on various prompt engineering techniques to enhance LLM performance in automated PIC design. The results reveal the challenges and potential of LLMs in the PIC design domain, offering insights into the key areas that require further research and development to optimize automation in this field. Our benchmark and evaluation code is available at https://github.com/PICDA/PICBench.

Related papers

• AMSnet-KG: A Netlist Dataset for LLM-based AMS Circuit Auto-Design Using Knowledge Graph RAG [15.61553255884534]
  Large language models (LLMs) have emerged as powerful tools for Electronic Design Automation (EDA) applications. This paper introduces AMSnet-KG, a dataset encompassing various AMS circuit schematics and netlists. We propose an automated AMS circuit generation framework that utilizes the comprehensive knowledge embedded in LLMs.
  arXiv  Detail & Related papers  (2024-11-07T02:49:53Z)
• LaMAGIC: Language-Model-based Topology Generation for Analog Integrated Circuits [17.002169206594793]
  We introduce LaMAGIC, a pioneering language model-based topology generation model. LaMAGIC can efficiently generate an optimized circuit design from the custom specification in a single pass. LaMAGIC achieves a success rate of up to 96% under a strict tolerance of 0.01.
  arXiv  Detail & Related papers  (2024-07-19T22:51:41Z)
• Benchmarking End-To-End Performance of AI-Based Chip Placement Algorithms [77.71341200638416]
  ChiPBench is a benchmark designed to evaluate the effectiveness of AI-based chip placement algorithms. We have gathered 20 circuits from various domains (e.g., CPU, GPU, and microcontrollers) for evaluation. Results show that even if intermediate metric of a single-point algorithm is dominant, the final PPA results are unsatisfactory.
  arXiv  Detail & Related papers  (2024-07-03T03:29:23Z)
• Digital ASIC Design with Ongoing LLMs: Strategies and Prospects [0.0]
  Large Language Models (LLMs) have been seen as a promising development, with the potential to automate the generation of Hardware Description Language (HDL) code. This paper presents targeted strategies to harness the capabilities of LLMs for digital ASIC design.
  arXiv  Detail & Related papers  (2024-04-25T05:16:57Z)
• LLM4EDA: Emerging Progress in Large Language Models for Electronic Design Automation [74.7163199054881]
  Large Language Models (LLMs) have demonstrated their capability in context understanding, logic reasoning and answer generation. We present a systematic study on the application of LLMs in the EDA field. We highlight the future research direction, focusing on applying LLMs in logic synthesis, physical design, multi-modal feature extraction and alignment of circuits.
  arXiv  Detail & Related papers  (2023-12-28T15:09:14Z)
• CktGNN: Circuit Graph Neural Network for Electronic Design Automation [67.29634073660239]
  This paper presents a Circuit Graph Neural Network (CktGNN) that simultaneously automates the circuit topology generation and device sizing. We introduce Open Circuit Benchmark (OCB), an open-sourced dataset that contains $10$K distinct operational amplifiers. Our work paves the way toward a learning-based open-sourced design automation for analog circuits.
  arXiv  Detail & Related papers  (2023-08-31T02:20:25Z)
• ChipGPT: How far are we from natural language hardware design [34.22592995908168]
  This work attempts to demonstrate an automated design environment that explores LLMs to generate hardware logic designs from natural language specifications. We present a scalable four-stage zero-code logic design framework based on LLMs without retraining or finetuning.
  arXiv  Detail & Related papers  (2023-05-23T12:54:02Z)
• The Basis of Design Tools for Quantum Computing: Arrays, Decision Diagrams, Tensor Networks, and ZX-Calculus [55.58528469973086]
  Quantum computers promise to efficiently solve important problems classical computers never will. A fully automated quantum software stack needs to be developed. This work provides a look "under the hood" of today's tools and showcases how these means are utilized in them, e.g., for simulation, compilation, and verification of quantum circuits.
  arXiv  Detail & Related papers  (2023-01-10T19:00:00Z)
• CodeRL: Mastering Code Generation through Pretrained Models and Deep Reinforcement Learning [92.36705236706678]
  "CodeRL" is a new framework for program synthesis tasks through pretrained LMs and deep reinforcement learning. During inference, we introduce a new generation procedure with a critical sampling strategy. For the model backbones, we extended the encoder-decoder architecture of CodeT5 with enhanced learning objectives.
  arXiv  Detail & Related papers  (2022-07-05T02:42:15Z)
• NAS-Count: Counting-by-Density with Neural Architecture Search [74.92941571724525]
  We automate the design of counting models with Neural Architecture Search (NAS) We introduce an end-to-end searched encoder-decoder architecture, Automatic Multi-Scale Network (AMSNet)
  arXiv  Detail & Related papers  (2020-02-29T09:18:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.