ast2vec: Utilizing Recursive Neural Encodings of Python Programs
- URL: http://arxiv.org/abs/2103.11614v1
- Date: Mon, 22 Mar 2021 06:53:52 GMT
- Title: ast2vec: Utilizing Recursive Neural Encodings of Python Programs
- Authors: Benjamin Paa{\ss}en and Jessica McBroom and Bryn Jeffries and Irena
Koprinska and Kalina Yacef
- Abstract summary: We present ast2vec, a neural network that maps Python syntax trees to vectors and back.
Ast2vec has been trained on almost half a million programs of novice programmers.
- Score: 3.088385631471295
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Educational datamining involves the application of datamining techniques to
student activity. However, in the context of computer programming, many
datamining techniques can not be applied because they expect vector-shaped
input whereas computer programs have the form of syntax trees. In this paper,
we present ast2vec, a neural network that maps Python syntax trees to vectors
and back, thereby facilitating datamining on computer programs as well as the
interpretation of datamining results. Ast2vec has been trained on almost half a
million programs of novice programmers and is designed to be applied across
learning tasks without re-training, meaning that users can apply it without any
need for (additional) deep learning. We demonstrate the generality of ast2vec
in three settings: First, we provide example analyses using ast2vec on a
classroom-sized dataset, involving visualization, student motion analysis,
clustering, and outlier detection, including two novel analyses, namely a
progress-variance-projection and a dynamical systems analysis. Second, we
consider the ability of ast2vec to recover the original syntax tree from its
vector representation on the training data and two further large-scale
programming datasets. Finally, we evaluate the predictive capability of a
simple linear regression on top of ast2vec, obtaining similar results to
techniques that work directly on syntax trees. We hope ast2vec can augment the
educational datamining toolbelt by making analyses of computer programs easier,
richer, and more efficient.
Related papers
- Knowledge Composition using Task Vectors with Learned Anisotropic Scaling [51.4661186662329]
We introduce aTLAS, an algorithm that linearly combines parameter blocks with different learned coefficients, resulting in anisotropic scaling at the task vector level.
We show that such linear combinations explicitly exploit the low intrinsicity of pre-trained models, with only a few coefficients being the learnable parameters.
We demonstrate the effectiveness of our method in task arithmetic, few-shot recognition and test-time adaptation, with supervised or unsupervised objectives.
arXiv Detail & Related papers (2024-07-03T07:54:08Z) - Distributive Pre-Training of Generative Modeling Using Matrix-Product
States [0.0]
We consider an alternative training scheme utilizing basic tensor network operations, e.g., summation and compression.
The training algorithm is based on compressing the superposition state constructed from all the training data in product state representation.
We benchmark the algorithm on the MNIST dataset and show reasonable results for generating new images and classification tasks.
arXiv Detail & Related papers (2023-06-26T15:46:08Z) - Retrieval-Augmented Reinforcement Learning [63.32076191982944]
We train a network to map a dataset of past experiences to optimal behavior.
The retrieval process is trained to retrieve information from the dataset that may be useful in the current context.
We show that retrieval-augmented R2D2 learns significantly faster than the baseline R2D2 agent and achieves higher scores.
arXiv Detail & Related papers (2022-02-17T02:44:05Z) - HyperSeed: Unsupervised Learning with Vector Symbolic Architectures [5.258404928739212]
This paper presents a novel unsupervised machine learning approach named Hyperseed.
It leverages Vector Symbolic Architectures (VSA) for fast learning a topology preserving feature map of unlabelled data.
The two distinctive novelties of the Hyperseed algorithm are 1) Learning from only few input data samples and 2) A learning rule based on a single vector operation.
arXiv Detail & Related papers (2021-10-15T20:05:43Z) - Towards Open-World Feature Extrapolation: An Inductive Graph Learning
Approach [80.8446673089281]
We propose a new learning paradigm with graph representation and learning.
Our framework contains two modules: 1) a backbone network (e.g., feedforward neural nets) as a lower model takes features as input and outputs predicted labels; 2) a graph neural network as an upper model learns to extrapolate embeddings for new features via message passing over a feature-data graph built from observed data.
arXiv Detail & Related papers (2021-10-09T09:02:45Z) - Network Support for High-performance Distributed Machine Learning [17.919773898228716]
We propose a system model that captures both learning nodes (that perform computations) and information nodes (that provide data)
We then formulate the problem of selecting (i) which learning and information nodes should cooperate to complete the learning task, and (ii) the number of iterations to perform.
We devise an algorithm, named DoubleClimb, that can find a 1+1/|I|-competitive solution with cubic worst-case complexity.
arXiv Detail & Related papers (2021-02-05T19:38:57Z) - Massively Parallel Graph Drawing and Representation Learning [13.736789987448466]
Graph embedding, i.e. converting the vertices of a graph into numerical vectors, is a data mining task of high importance.
We propose MulticoreGEMPE, an information-theoretic method which can generate low and high-dimensional vectors.
arXiv Detail & Related papers (2020-11-06T17:18:14Z) - One-step regression and classification with crosspoint resistive memory
arrays [62.997667081978825]
High speed, low energy computing machines are in demand to enable real-time artificial intelligence at the edge.
One-step learning is supported by simulations of the prediction of the cost of a house in Boston and the training of a 2-layer neural network for MNIST digit recognition.
Results are all obtained in one computational step, thanks to the physical, parallel, and analog computing within the crosspoint array.
arXiv Detail & Related papers (2020-05-05T08:00:07Z) - Einsum Networks: Fast and Scalable Learning of Tractable Probabilistic
Circuits [99.59941892183454]
We propose Einsum Networks (EiNets), a novel implementation design for PCs.
At their core, EiNets combine a large number of arithmetic operations in a single monolithic einsum-operation.
We show that the implementation of Expectation-Maximization (EM) can be simplified for PCs, by leveraging automatic differentiation.
arXiv Detail & Related papers (2020-04-13T23:09:15Z) - ProGraML: Graph-based Deep Learning for Program Optimization and
Analysis [16.520971531754018]
We introduce ProGraML, a graph-based program representation for machine learning.
ProGraML achieves an average 94.0 F1 score, significantly outperforming the state-of-the-art approaches.
We then apply our approach to two high-level tasks - heterogeneous device mapping and program classification - setting new state-of-the-art performance in both.
arXiv Detail & Related papers (2020-03-23T20:27:00Z) - Multi-layer Optimizations for End-to-End Data Analytics [71.05611866288196]
We introduce Iterative Functional Aggregate Queries (IFAQ), a framework that realizes an alternative approach.
IFAQ treats the feature extraction query and the learning task as one program given in the IFAQ's domain-specific language.
We show that a Scala implementation of IFAQ can outperform mlpack, Scikit, and specialization by several orders of magnitude for linear regression and regression tree models over several relational datasets.
arXiv Detail & Related papers (2020-01-10T16:14:44Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.