That Chip Has Sailed: A Critique of Unfounded Skepticism Around AI for Chip Design

• URL: http://arxiv.org/abs/2411.10053v1
• Date: Fri, 15 Nov 2024 09:11:10 GMT
• Title: That Chip Has Sailed: A Critique of Unfounded Skepticism Around AI for Chip Design
• Authors: Anna Goldie, Azalia Mirhoseini, Jeff Dean,
• Abstract summary: In 2020, we introduced a deep reinforcement learning method capable of generating superhuman chip layouts. A non-peer-reviewed paper at ISPD 2023 questioned its performance claims, despite failing to run our method as described in Nature. We publish this response to ensure that no one is wrongly discouraged from innovating in this impactful area.
• Score: 6.383127282050037
• License:
• Abstract: In 2020, we introduced a deep reinforcement learning method capable of generating superhuman chip layouts, which we then published in Nature and open-sourced on GitHub. AlphaChip has inspired an explosion of work on AI for chip design, and has been deployed in state-of-the-art chips across Alphabet and extended by external chipmakers. Even so, a non-peer-reviewed invited paper at ISPD 2023 questioned its performance claims, despite failing to run our method as described in Nature. For example, it did not pre-train the RL method (removing its ability to learn from prior experience), used substantially fewer compute resources (20x fewer RL experience collectors and half as many GPUs), did not train to convergence (standard practice in machine learning), and evaluated on test cases that are not representative of modern chips. Recently, Igor Markov published a meta-analysis of three papers: our peer-reviewed Nature paper, the non-peer-reviewed ISPD paper, and Markov's own unpublished paper (though he does not disclose that he co-authored it). Although AlphaChip has already achieved widespread adoption and impact, we publish this response to ensure that no one is wrongly discouraged from innovating in this impactful area.

Related papers

• NLLG Quarterly arXiv Report 09/24: What are the most influential current AI Papers? [21.68589129842815]
  The NLLG arXiv reports assist in navigating the rapidly evolving landscape of NLP and AI research across cs.CL, cs.CV, cs.AI, and cs.LG categories. This fourth installment captures a transformative period in AI history - from January 1, 2023, following ChatGPT's debut, through September 30, 2024. Our analysis reveals substantial new developments in the field - with 45% of the top 40 most-cited papers being new entries since our last report.
  arXiv  Detail & Related papers  (2024-12-02T22:10:38Z)
• Benchmarking End-To-End Performance of AI-Based Chip Placement Algorithms [77.71341200638416]
  ChiPBench is a benchmark designed to evaluate the effectiveness of AI-based chip placement algorithms. We have gathered 20 circuits from various domains (e.g., CPU, GPU, and microcontrollers) for evaluation. Results show that even if intermediate metric of a single-point algorithm is dominant, the final PPA results are unsatisfactory.
  arXiv  Detail & Related papers  (2024-07-03T03:29:23Z)
• Theoretically Achieving Continuous Representation of Oriented Bounding Boxes [64.15627958879053]
  This paper endeavors to completely solve the issue of discontinuity in Oriented Bounding Box representation. We propose a novel representation method called Continuous OBB (COBB) which can be readily integrated into existing detectors. For fairness and transparency of experiments, we have developed a modularized benchmark based on the open-source deep learning framework Jittor's detection toolbox JDet for OOD evaluation.
  arXiv  Detail & Related papers  (2024-02-29T09:27:40Z)
• Simulation of IBM's kicked Ising experiment with Projected Entangled Pair Operator [71.10376783074766]
  We perform classical simulations of the 127-qubit kicked Ising model, which was recently emulated using a quantum circuit with error mitigation. Our approach is based on the projected entangled pair operator (PEPO) in the Heisenberg picture. We develop a Clifford expansion theory to compute exact expectation values and use them to evaluate algorithms.
  arXiv  Detail & Related papers  (2023-08-06T10:24:23Z)
• A LLM Assisted Exploitation of AI-Guardian [57.572998144258705]
  We evaluate the robustness of AI-Guardian, a recent defense to adversarial examples published at IEEE S&P 2023. We write none of the code to attack this model, and instead prompt GPT-4 to implement all attack algorithms following our instructions and guidance. This process was surprisingly effective and efficient, with the language model at times producing code from ambiguous instructions faster than the author of this paper could have done.
  arXiv  Detail & Related papers  (2023-07-20T17:33:25Z)
• ChiPFormer: Transferable Chip Placement via Offline Decision Transformer [35.69382855465161]
  reinforcement learning can improve human performance in chip placement. ChiPFormer enables learning a transferable placement policy from fixed offline data. ChiPFormer achieves significantly better placement quality while reducing the runtime by 10x.
  arXiv  Detail & Related papers  (2023-06-26T14:59:56Z)
• The False Dawn: Reevaluating Google's Reinforcement Learning for Chip Macro Placement [1.4803764446062861]
  Reinforcement learning for physical design of silicon chips in a Google 2021 Nature paper stirred controversy due to poorly documented claims. We show that Google RL lags behind (i) human designers, (ii) a well-known algorithm (Simulated Annealing), and (iii) generally-available commercial software, while being slower. Crosschecked data indicate that the integrity of the Nature paper is substantially undermined owing to errors in conduct, analysis and reporting.
  arXiv  Detail & Related papers  (2023-06-16T05:32:24Z)
• Tricking AI chips into Simulating the Human Brain: A Detailed Performance Analysis [0.5354801701968198]
  We evaluate multiple, cutting-edge AI-chips (Graphcore IPU, GroqChip, Nvidia GPU with inferior Cores and Google TPU) for brain simulation. Our performance analysis reveals that the simulation problem maps extremely well onto the GPU and TPU architectures. The GroqChip outperforms both platforms for small networks but, due to implementing some floating-point operations at reduced accuracy, is found not yet usable for brain simulation.
  arXiv  Detail & Related papers  (2023-01-31T13:51:37Z)
• Self-PU: Self Boosted and Calibrated Positive-Unlabeled Training [118.10946662410639]
  We propose a novel Self-PU learning framework, which seamlessly integrates PU learning and self-training. Self-PU highlights three "self"-oriented building blocks: a self-paced training algorithm that adaptively discovers and augments confident examples as the training proceeds. We study a real-world application of PU learning, i.e., classifying brain images of Alzheimer's Disease.
  arXiv  Detail & Related papers  (2020-06-22T17:53:59Z)
• Chip Placement with Deep Reinforcement Learning [40.952111701288125]
  We present a learning-based approach to chip placement. Unlike prior methods, our approach has the ability to learn from past experience and improve over time. In under 6 hours, our method can generate placements that are superhuman or comparable on modern accelerator netlists.
  arXiv  Detail & Related papers  (2020-04-22T17:56:07Z)
• Robust Pruning at Initialization [61.30574156442608]
  A growing need for smaller, energy-efficient, neural networks to be able to use machine learning applications on devices with limited computational resources. For Deep NNs, such procedures remain unsatisfactory as the resulting pruned networks can be difficult to train and, for instance, they do not prevent one layer from being fully pruned.
  arXiv  Detail & Related papers  (2020-02-19T17:09:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.