Antecedent Predictions Are More Important Than You Think: An Effective
Method for Tree-Based Code Generation
- URL: http://arxiv.org/abs/2208.09998v3
- Date: Mon, 17 Jul 2023 22:36:57 GMT
- Title: Antecedent Predictions Are More Important Than You Think: An Effective
Method for Tree-Based Code Generation
- Authors: Yihong Dong, Ge Li, Xue Jiang, and Zhi Jin
- Abstract summary: Existing Seq2Tree methods tend to treat both antecedent predictions and subsequent predictions equally.
We propose Antecedentd Prioritized Tree-based code generation model called APT.
With better predictions, APT significantly improves the performance.
- Score: 25.51290127187619
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Code generation focuses on the automatic conversion of natural language (NL)
utterances into code snippets. The sequence-to-tree (Seq2Tree) approaches are
proposed for code generation, with the guarantee of the grammatical correctness
of the generated code, which generate the subsequent Abstract Syntax Tree (AST)
node relying on antecedent predictions of AST nodes. Existing Seq2Tree methods
tend to treat both antecedent predictions and subsequent predictions equally.
However, under the AST constraints, it is difficult for Seq2Tree models to
produce the correct subsequent prediction based on incorrect antecedent
predictions. Thus, antecedent predictions ought to receive more attention than
subsequent predictions. To this end, in this paper, we propose an effective
method, named Antecedent Prioritized (AP) Loss, that helps the model attach
importance to antecedent predictions by exploiting the position information of
the generated AST nodes. We design an AST-to-Vector (AST2Vec) method, that maps
AST node positions to two-dimensional vectors, to model the position
information of AST nodes. To evaluate the effectiveness of our proposed loss,
we implement and train an Antecedent Prioritized Tree-based code generation
model called APT. With better antecedent predictions and accompanying
subsequent predictions, APT significantly improves the performance. We conduct
extensive experiments on four benchmark datasets, and the experimental results
demonstrate the superiority and generality of our proposed method.
Related papers
- PrevPredMap: Exploring Temporal Modeling with Previous Predictions for Online Vectorized HD Map Construction [9.32290307534907]
PrevPredMap is a pioneering temporal modeling framework that leverages previous predictions for constructing online vectorized HD maps.
The framework achieves state-of-the-art performance on the nuScenes and Argoverse2 datasets.
arXiv Detail & Related papers (2024-07-24T15:58:24Z) - Graph-Structured Speculative Decoding [52.94367724136063]
Speculative decoding has emerged as a promising technique to accelerate the inference of Large Language Models.
We introduce an innovative approach utilizing a directed acyclic graph (DAG) to manage the drafted hypotheses.
We observe a remarkable speedup of 1.73$times$ to 1.96$times$, significantly surpassing standard speculative decoding.
arXiv Detail & Related papers (2024-07-23T06:21:24Z) - Towards Human-AI Complementarity with Predictions Sets [14.071862670474832]
Decision support systems based on prediction sets have proven to be effective at helping human experts solve classification tasks.
We show that the prediction sets constructed using conformal prediction are, in general, suboptimal in terms of average accuracy.
We introduce a greedy algorithm that, for a large class of expert models and non-optimal scores, is guaranteed to find prediction sets that provably offer equal or greater performance.
arXiv Detail & Related papers (2024-05-27T18:00:00Z) - A positive feedback method based on F-measure value for Salient Object
Detection [1.9249287163937976]
This paper proposes a positive feedback method based on F-measure value for salient object detection (SOD)
Our proposed method takes an image to be detected and inputs it into several existing models to obtain their respective prediction maps.
Experimental results on five publicly available datasets show that our proposed positive feedback method outperforms the latest 12 methods in five evaluation metrics for saliency map prediction.
arXiv Detail & Related papers (2023-04-28T04:05:13Z) - Social Interpretable Tree for Pedestrian Trajectory Prediction [75.81745697967608]
We propose a tree-based method, termed as Social Interpretable Tree (SIT), to address this multi-modal prediction task.
A path in the tree from the root to leaf represents an individual possible future trajectory.
Despite the hand-crafted tree, the experimental results on ETH-UCY and Stanford Drone datasets demonstrate that our method is capable of matching or exceeding the performance of state-of-the-art methods.
arXiv Detail & Related papers (2022-05-26T12:18:44Z) - Efficient and Differentiable Conformal Prediction with General Function
Classes [96.74055810115456]
We propose a generalization of conformal prediction to multiple learnable parameters.
We show that it achieves approximate valid population coverage and near-optimal efficiency within class.
Experiments show that our algorithm is able to learn valid prediction sets and improve the efficiency significantly.
arXiv Detail & Related papers (2022-02-22T18:37:23Z) - Complex Event Forecasting with Prediction Suffix Trees: Extended
Technical Report [70.7321040534471]
Complex Event Recognition (CER) systems have become popular in the past two decades due to their ability to "instantly" detect patterns on real-time streams of events.
There is a lack of methods for forecasting when a pattern might occur before such an occurrence is actually detected by a CER engine.
We present a formal framework that attempts to address the issue of Complex Event Forecasting.
arXiv Detail & Related papers (2021-09-01T09:52:31Z) - CASTLE: Regularization via Auxiliary Causal Graph Discovery [89.74800176981842]
We introduce Causal Structure Learning (CASTLE) regularization and propose to regularize a neural network by jointly learning the causal relationships between variables.
CASTLE efficiently reconstructs only the features in the causal DAG that have a causal neighbor, whereas reconstruction-based regularizers suboptimally reconstruct all input features.
arXiv Detail & Related papers (2020-09-28T09:49:38Z) - A Hybrid Two-layer Feature Selection Method Using GeneticAlgorithm and
Elastic Net [6.85316573653194]
This paper presents a new hybrid two-layer feature selection approach that combines a wrapper and an embedded method.
The Genetic Algorithm(GA) has been adopted as a wrapper to search for the optimal subset of predictors.
A second layer is added to the proposed method to eliminate any remaining redundant/irrelevant predictors to improve the prediction accuracy.
arXiv Detail & Related papers (2020-01-30T05:01:30Z) - ProphetNet: Predicting Future N-gram for Sequence-to-Sequence
Pre-training [85.35910219651572]
We present a new sequence-to-sequence pre-training model called ProphetNet.
It introduces a novel self-supervised objective named future n-gram prediction.
We conduct experiments on CNN/DailyMail, Gigaword, and SQuAD 1.1 benchmarks for abstractive summarization and question generation tasks.
arXiv Detail & Related papers (2020-01-13T05:12:38Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.