TF-Coder: Program Synthesis for Tensor Manipulations

• URL: http://arxiv.org/abs/2003.09040v4
• Date: Thu, 7 Apr 2022 22:17:19 GMT
• Title: TF-Coder: Program Synthesis for Tensor Manipulations
• Authors: Kensen Shi, David Bieber, Rishabh Singh
• Abstract summary: We present a tool called TF-Coder for programming by example in pruning. We train models to predict operations from features of the input and output tensors and natural language descriptions of tasks. TF-Coder solves 63 of 70 real-world tasks within 5 minutes, sometimes finding simpler solutions in less time compared to experienced human programmers.
• Score: 29.46838583290554
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The success and popularity of deep learning is on the rise, partially due to powerful deep learning frameworks such as TensorFlow and PyTorch that make it easier to develop deep learning models. However, these libraries also come with steep learning curves, since programming in these frameworks is quite different from traditional imperative programming with explicit loops and conditionals. In this work, we present a tool called TF-Coder for programming by example in TensorFlow. TF-Coder uses a bottom-up weighted enumerative search, with value-based pruning of equivalent expressions and flexible type- and value-based filtering to ensure that expressions adhere to various requirements imposed by the TensorFlow library. We train models to predict TensorFlow operations from features of the input and output tensors and natural language descriptions of tasks, to prioritize relevant operations during search. TF-Coder solves 63 of 70 real-world tasks within 5 minutes, sometimes finding simpler solutions in less time compared to experienced human programmers.

Related papers

• SparseCoder: Identifier-Aware Sparse Transformer for File-Level Code Summarization [51.67317895094664]
  This paper studies file-level code summarization, which can assist programmers in understanding and maintaining large source code projects. We propose SparseCoder, an identifier-aware sparse transformer for effectively handling long code sequences.
  arXiv  Detail & Related papers  (2024-01-26T09:23:27Z)
• CodeGen2: Lessons for Training LLMs on Programming and Natural Languages [116.74407069443895]
  We unify encoder and decoder-based models into a single prefix-LM. For learning methods, we explore the claim of a "free lunch" hypothesis. For data distributions, the effect of a mixture distribution and multi-epoch training of programming and natural languages on model performance is explored.
  arXiv  Detail & Related papers  (2023-05-03T17:55:25Z)
• Trieste: Efficiently Exploring The Depths of Black-box Functions with TensorFlow [50.691232400959656]
  Trieste is an open-source Python package for Bayesian optimization and active learning. Our library enables the plug-and-play of popular models within sequential decision-making loops.
  arXiv  Detail & Related papers  (2023-02-16T17:21:49Z)
• ExAIS: Executable AI Semantics [4.092001692194709]
  Neural networks can be regarded as a new programming paradigm, i.e., instead of building ever-more complex programs through (often informal) logical reasoning in the programmers' mind, complex 'AI' systems are built by optimising generic neural network models with big data. In this new paradigm, AI frameworks such as PyTorch play a key role, which is as essential as the compiler for traditional programs. It is known that the lack of a proper semantics for programming languages (such as C), i.e., a correctness specification for compilers, has contributed to many problematic program behaviours and security issues
  arXiv  Detail & Related papers  (2022-02-20T17:33:34Z)
• OneFlow: Redesign the Distributed Deep Learning Framework from Scratch [17.798586916628174]
  OneFlow is a novel distributed training framework based on an SBP (split, broadcast and partial-value) abstraction and the actor model. SBP enables much easier programming of data parallelism and model parallelism than existing frameworks. OneFlow outperforms many well-known customized libraries built on top of the state-of-the-art frameworks.
  arXiv  Detail & Related papers  (2021-10-28T11:32:14Z)
• TensorFlow ManOpt: a library for optimization on Riemannian manifolds [0.3655021726150367]
  The adoption of neural networks and deep learning in non-Euclidean domains has been hindered until recently by the lack of scalable and efficient learning frameworks. We attempt to bridge this gap by proposing ManOpt, a Python library for optimization on Riemannian in terms of machine learning models. The library is designed with the aim for a seamless integration with the ecosystem, targeting not only research, but also streamlining production machine learning pipelines.
  arXiv  Detail & Related papers  (2021-05-27T10:42:09Z)
• TensorGP -- Genetic Programming Engine in TensorFlow [1.1470070927586016]
  We investigate the benefits of applying data vectorization and fitness caching methods to domain evaluation in Genetic Programming. Our performance benchmarks demonstrate that performance gains of up to two orders of magnitude can be achieved on a parallel approach running on dedicated hardware.
  arXiv  Detail & Related papers  (2021-03-12T20:19:37Z)
• Self Normalizing Flows [65.73510214694987]
  We propose a flexible framework for training normalizing flows by replacing expensive terms in the gradient by learned approximate inverses at each layer. This reduces the computational complexity of each layer's exact update from $mathcalO(D3)$ to $mathcalO(D2)$. We show experimentally that such models are remarkably stable and optimize to similar data likelihood values as their exact gradient counterparts.
  arXiv  Detail & Related papers  (2020-11-14T09:51:51Z)
• GraphCodeBERT: Pre-training Code Representations with Data Flow [97.00641522327699]
  We present GraphCodeBERT, a pre-trained model for programming language that considers the inherent structure of code. We use data flow in the pre-training stage, which is a semantic-level structure of code that encodes the relation of "where-the-value-comes-from" between variables. We evaluate our model on four tasks, including code search, clone detection, code translation, and code refinement.
  arXiv  Detail & Related papers  (2020-09-17T15:25:56Z)
• An Advance on Variable Elimination with Applications to Tensor-Based Computation [11.358487655918676]
  We present new results on the classical algorithm of variable elimination, which underlies many algorithms including for probabilistic inference. The results relate to exploiting functional dependencies, allowing one to perform inference and learning efficiently on models that have very large treewidth.
  arXiv  Detail & Related papers  (2020-02-21T14:17:44Z)
• Multi-layer Optimizations for End-to-End Data Analytics [71.05611866288196]
  We introduce Iterative Functional Aggregate Queries (IFAQ), a framework that realizes an alternative approach. IFAQ treats the feature extraction query and the learning task as one program given in the IFAQ's domain-specific language. We show that a Scala implementation of IFAQ can outperform mlpack, Scikit, and specialization by several orders of magnitude for linear regression and regression tree models over several relational datasets.
  arXiv  Detail & Related papers  (2020-01-10T16:14:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.