PyGlove: Efficiently Exchanging ML Ideas as Code

• URL: http://arxiv.org/abs/2302.01918v1
• Date: Fri, 3 Feb 2023 18:52:09 GMT
• Title: PyGlove: Efficiently Exchanging ML Ideas as Code
• Authors: Daiyi Peng, Xuanyi Dong, Esteban Real, Yifeng Lu, Quoc V. Le
• Abstract summary: PyGlove represents ideas as symbolic rule-based patches, enabling researchers to write down the rules for models they have not seen. This permits a network effect among teams: at once, any team can issue patches to all other teams.
• Score: 81.80955202879686
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The increasing complexity and scale of machine learning (ML) has led to the need for more efficient collaboration among multiple teams. For example, when a research team invents a new architecture like "ResNet," it is desirable for multiple engineering teams to adopt it. However, the effort required for each team to study and understand the invention does not scale well with the number of teams or inventions. In this paper, we present an extension of our PyGlove library to easily and scalably share ML ideas. PyGlove represents ideas as symbolic rule-based patches, enabling researchers to write down the rules for models they have not seen. For example, an inventor can write rules that will "add skip-connections." This permits a network effect among teams: at once, any team can issue patches to all other teams. Such a network effect allows users to quickly surmount the cost of adopting PyGlove by writing less code quicker, providing a benefit that scales with time. We describe the new paradigm of organizing ML through symbolic patches and compare it to existing approaches. We also perform a case study of a large codebase where PyGlove led to an 80% reduction in the number of lines of code.

Related papers

• Steering Large Language Models between Code Execution and Textual Reasoning [22.279107036500083]
  Textual reasoning has inherent limitations in solving tasks with challenges in math, logics, optimization, and searching. The recently released OpenAI GPT Code Interpreter and multi-agent frameworks such as AutoGen have demonstrated remarkable proficiency of integrating code generation and execution. We propose three methods to better steer LLM code/text generation and achieve a notable improvement.
  arXiv  Detail & Related papers  (2024-10-04T15:44:47Z)
• AI Coders Are Among Us: Rethinking Programming Language Grammar Towards Efficient Code Generation [14.831115535710692]
  We propose the concept of AI-oriented grammar. This aims to represent code in a way that better suits the working mechanism of AI models. Code written with AI-oriented grammar discards formats and uses a minimum number of tokens.
  arXiv  Detail & Related papers  (2024-04-25T04:46:02Z)
• Open Ad Hoc Teamwork with Cooperative Game Theory [28.605478081031215]
  Ad hoc teamwork poses a challenging problem, requiring the design of an agent to collaborate with teammates without prior coordination or joint training. One promising solution is leveraging the generalizability of graph neural networks to handle an unrestricted number of agents. We propose a novel algorithm named CIAO, based on the game's framework, with additional provable implementation tricks that can facilitate learning.
  arXiv  Detail & Related papers  (2024-02-23T11:04:33Z)
• torchgfn: A PyTorch GFlowNet library [56.071033896777784]
  torchgfn is a PyTorch library that aims to address this need. It provides users with a simple API for environments and useful abstractions for samplers and losses.
  arXiv  Detail & Related papers  (2023-05-24T00:20:59Z)
• Low-code LLM: Graphical User Interface over Large Language Models [115.08718239772107]
  This paper introduces a novel human-LLM interaction framework, Low-code LLM. It incorporates six types of simple low-code visual programming interactions to achieve more controllable and stable responses. We highlight three advantages of the low-code LLM: user-friendly interaction, controllable generation, and wide applicability.
  arXiv  Detail & Related papers  (2023-04-17T09:27:40Z)
• PAL: Program-aided Language Models [112.94785609781503]
  We present Program-Aided Language models (PaL) to understand natural language problems. PaL offloads the solution step to a programmatic runtime such as a Python interpreter. We set new state-of-the-art results in all 12 benchmarks.
  arXiv  Detail & Related papers  (2022-11-18T18:56:13Z)
• pyGSL: A Graph Structure Learning Toolkit [14.000763778781547]
  pyGSL is a Python library that provides efficient implementations of state-of-the-art graph structure learning models. pyGSL is written in GPU-friendly ways, allowing one to scale to much larger network tasks.
  arXiv  Detail & Related papers  (2022-11-07T14:23:10Z)
• PyGlove: Symbolic Programming for Automated Machine Learning [88.15565138144042]
  We introduce a new way of programming AutoML based on symbolic programming. Under this paradigm, ML programs are mutable, thus can be manipulated easily by another program. We show that PyGlove users can easily convert a static program into a search space, quickly iterate on the search spaces and search algorithms, and craft complex search flows.
  arXiv  Detail & Related papers  (2021-01-21T19:05:44Z)
• COSEA: Convolutional Code Search with Layer-wise Attention [90.35777733464354]
  We propose a new deep learning architecture, COSEA, which leverages convolutional neural networks with layer-wise attention to capture the code's intrinsic structural logic. COSEA can achieve significant improvements over state-of-the-art methods on code search tasks.
  arXiv  Detail & Related papers  (2020-10-19T13:53:38Z)
• PyTorch Metric Learning [37.03614011735927]
  PyTorch Metric Learning is an open source library that aims to remove this barrier for both researchers and practitioners. The modular and flexible design allows users to easily try out different combinations of algorithms in their existing code. It also comes with complete train/test, for users who want results fast.
  arXiv  Detail & Related papers  (2020-08-20T19:08:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.