Learning from source code usually requires a large amount of labeled data.
Despite the possible scarcity of labeled data, the trained model is highly
task-specific and lacks transferability to different tasks. In this work, we
present effective pre-training strategies on top of a novel graph-based code
representation, to produce universal representations for code. Specifically,
our graph-based representation captures important semantics between code
elements (e.g., control flow and data flow). We pre-train graph neural networks
on the representation to extract universal code properties. The pre-trained
model then enables the possibility of fine-tuning to support various downstream
applications. We evaluate our model on two real-world datasets -- spanning over
30M Java methods and 770K Python methods. Through visualization, we reveal
discriminative properties in our universal code representation. By comparing
multiple benchmarks, we demonstrate that the proposed framework achieves
state-of-the-art results on method name prediction and code graph link
prediction.
In this work, we present effective pre-training strategies on top of a novel graph-based code representation, to produce universal representations for code.
Through visualization, we reveal discriminative properties in our universal code representation.
可視化により、普遍的なコード表現における識別特性を明らかにする。
0.58
By comparing multiple benchmarks, we demonstrate that the proposed framework achieves state-of-the-art results on method name prediction and code graph link prediction.
Introduction 1 Analysis of software using machine learning approaches has several important applications such as identifying code defects [1], improving code search [2], and improving developer productivity [3].
The σ0/σ1 graph is defined in Section 3, and the pre-training signals are defined in Section 4.
σ0/σ1グラフはセクション3で定義され、前訓練信号はセクション4で定義されます。
0.71
trees (ASTs) [5] that highlight structural and content-related details in code.
tree (ASTs) [5] コードの構造と内容に関する詳細を強調します。
0.84
However, some critical relations (e g , control flow and data flow), which often impact machine learning models’ success in abstracting code information, are not available in trees.
Depending on the type of symbolic representation used, the approaches for generating the neural code representations are either sequence-based [3,7] or graph-based [4,6] neural network models.
In this work, we touch upon all the three aspects of ML-based code analysis: symbolic code representation, task-independent neural code representation, and task-specific learning.
For symbolic code representation, we explore two alternatives and show that symbolic code representation (called σ0/σ1 graph) which captures richer relations leads to better performance in downstream tasks.
And, finally, we explore two tasks to demonstrate the effectiveness of the learned representations: method name prediction (for Python and Java) and link prediction (for Java).
Our proposed method consistently improves the prediction accuracy across all experiments.
提案手法は全実験で一貫して予測精度を向上させる。
0.70
To summarize, the contributions of this work are as follows: – We introduce a fine-grained symbolic graph representation for source code, and adapt to 29M Java methods collected from GitHub.
– We present a pre-training framework that leverages the graph-based code representations to produce universal code representations, supporting various downstream tasks via fine-tuning.
– We combine the graph-based representation and the pre-training strategies to go beyond code pre-training with sequence- and tree-based representations.
RGCN’s update rule is given by (cid:88) n∈N r r∈R i is the neighbor set of node i under edge type r, ci,r is a normalizer (we i | as suggested in [10]), h(l) is the hidden representation of node i i r are learnable parameters, and φ(·) is any nonlinear activation (the representation
RGCN の更新規則は (cid:88) n∈N r r∈R i がエッジ型 r の下のノード i の隣接集合、 ci,r が正規化子 (we i | が [10] で示される)、h(l) がノード i i r の隠れた表現が学習可能なパラメータ、φ(...) が任意の非線形活性化(表現)によって与えられる。
0.85
where N r i use ci,r = |N r at layer l, W(l) function.
ここで N r i は層 l, W(l) で ci,r = |N r を用いる。
0.83
Usually, h(0) i at the last layer) is used as the final representations.
通常、最後の層におけるh(0) i) は最終表現として使われる。
0.82
is initialized as node features, and h(L) i
is initialized as node features, and h(L) i
0.85
h(l+1) i = φ
h(l+1) i = φ
0.87
1 ci,r h(l) n W(l) r
1 ci,r h(l) n W(l) r
0.85
2.2 Pre-training for GNNs and for Source Code Recently, there is a rising interest in pre-training GNNs to model graph data [11,12].
To pre-train GNNs, most works encourage GNNs to capture graph structure information (e g , graph motif) and graph node information (e g , node feature).
We call our code graphs as σ0 graphs and σ1 graphs.
コードグラフをσ0グラフとσ1グラフと呼びます。
0.82
The σ0 graphs are related to classical Program Dependence Graphs (PDGs) [17].
σ0グラフは古典的プログラム依存グラフ(PDG) [17] と関連している。
0.88
The σ1 graphs build upon the σ0 graphs and include additional syntactic and semantic information (detailed in section 3.2).
σ1グラフはσ0グラフの上に構築され、追加構文および意味情報を含む(セクション3.2)。
0.78
Our experiments show that ML models using the σ1 graphs achieve better prediction accuracy than using the σ0 graphs.
実験の結果,σ1グラフを用いたMLモデルはσ0グラフよりも予測精度がよいことがわかった。
0.75
3.1 The σ0 Graph The σ0 graphs, which relate to PDGs, are used for tasks such as detection of security vulnerabilities and identification of concurrency issues.
In σ0 graphs, nodes represent different kinds of program elements including data and operations; edges represent different kinds of control flow and data flow between program elements.
Examples of edge types include parameter edges which indicate the data flow into operations and throw edges which represent the control flows when exceptions are thrown.
3.2 The σ1 Graph One limitation of σ0 graph is that the downstream modules perform additional analysis such as control dependence and aliasing is not reflected in the σ0 graph explicitly and requires machine learning models to infer it.
Higher-order semantic relations such as variable usage (e g , FirstUse/LastUse), node aliasing, and control dependence are also included as graph edges.
The instantiation of the GNN encoder is given in the experiment.
この実験では、GNNエンコーダのインスタンス化を行う。
0.65
英語（論文から抽出）
日本語訳
スコア
6 L. Liu et al
6 L. Liu et al
0.92
(cid:88) v,v(cid:48)∈V
(cid:88) v,v(cid:48)∈V
0.81
(cid:16) (cid:16)−y × ht vWt,t(cid:48)
(cid:16) (cid:16)−y×htvwt,t(cid:48)
0.84
(cid:17)(cid:17)
(cid:17)(cid:17)
0.75
, 4.1 Pre-training Signals Metapath Random Walk (MRW) signal.
, 4.1 事前学習信号メタパスランダムウォーク(MRW)信号
0.81
A MRW is a random path that follows a sequence of edge types.
MRWは、エッジ型のシーケンスに従うランダムパスである。
0.62
We assume node pairs in the same MRW are proximal to each other; accordingly, they should have similar embeddings.
同じMRWのノードペアは互いに近縁であると仮定し、それ故に同様の埋め込みを持つべきです。
0.69
For example, for a MRW with nodes [Collection.iterator( ), Iterator, Iterator.hasNext()] and edges [definition, receiver], nodes Iterator and Iterator.hasNext() are expected to have similar embeddings.
LM RW = (1) v(cid:48) are embeddings for nodes v and v(cid:48) with node types t and t(cid:48).
LM RW = (1) v(cid:48) は、ノード型 t と t(cid:48) を持つノード v と v(cid:48) の埋め込みである。
0.79
where ht Wt,t(cid:48) is a diagonal matrix weighing the similarity between different node types.
ht Wt,t(cid:48) は異なるノードタイプ間の類似性を測る対角行列である。
0.82
y equals 1 as positive pairs if v and v(cid:48) are in the same MRW, otherwise y equals -1 as negative pairs.
v と v(cid:48) が同じ MRW にある場合、y は正対 1 であり、そうでなければ y は負対 -1 と等しい。
0.74
During training, we sample 5 negative pairs per positive pair.
トレーニング中、ポジティブペアあたり5つのネガティブペアをサンプリングします。
0.59
Heterogeneous Information Maximization (HIM) signal.
Heterogeneous Information Maximization (HIM) 信号。
0.77
Nodes of the same type should reside in some shared embedding space, encoding their similarity.
同じタイプのノードは、その類似性をエンコードする共有埋め込みスペースに置かれるべきである。
0.70
On the other hand, nodes of different types, such as control nodes and data nodes, ought to have discriminative embedding space as they are semantically different.
However, standard GNNs fail to do so with only local message propagation.
しかし、標準のGNNは、ローカルメッセージの伝搬だけでは失敗します。
0.76
v Wst)(cid:1)(cid:17) (cid:88) (cid:88) v Wst)(cid:1) + log(cid:0)1 − φ(˜h(cid:62) log(cid:0)φ(h(cid:62) Following [8], we use a HIM signal to encode these properties: LHIM = (2) (cid:80) t∈T v∈V t v∈V t hv is a global summary of nodes typed t, φ(·) is a sigmoid Here, st = 1|V t| function, and φ(h(cid:62) v Wst) quantifies the closeness between a node embedding hv and a global summary st.
v wst)(cid:17) (cid:88) (cid:88) (cid:88) v wst)(cid:1) + log(cid:0)1 − φ( sh(cid:62) log(cid:0) φ(h(cid:62) following [8], we use a him signal to encoding these properties: lhim = (2) (cid:80) tservlett v vسv t hv is a global summary of nodes typed t, φ(·) is a sigmoid, st = 1|v t| function, φ(h(cid:62) v wst) is the closeness between a node embedded hv to a global summary.
0.94
Negative samples ˜hv are obtained by first row-wise shuffling input node features then propagating through the GNN encoder [8].
Such structural patterns can be captured in graph motifs, see Fig 3.
このような構造パターンはグラフモチーフで捉えることができる。
0.65
With this observation, we pretrain GNNs with MT signal to generate structureFig.
この観察により、MT信号でGNNを事前訓練し、StructureFigを生成する。
0.64
3. Motifs sized 3 and 4. aware node embeddings for code graphs.
3. モチーフサイズは3と4で、コードグラフ用のノード埋め込みを意識する。
0.73
Formally, we aim to recover the ground truth motif around each node, mv, from the node embedding hv using an approximator fM T (·) (we use a two-layer MLP),
形式的には、近似器 fM T (·) を用いてhv を埋め込んだノードから、各ノード mv の周りの基底的真理モチーフを復元することを目的としている(2層 MLP を用いる)。
0.71
(cid:16) .
(cid:16) .
0.82
LM T = ||mv − fM T (hv)||2 2.
LM T = ||mv − fM T (hv)||2。
0.87
(3) (cid:88) v∈V
(3) (cid:88) v∈V
0.75
The ground truth mv is obtained using a fast motif extraction method [18].
接地真理mvは高速モチーフ抽出法[18]を用いて得られる。
0.68
Node Tying (NT) signal.
ノードタイリング(NT)信号。
0.78
The corpus-level graph (Cf.
コーパスレベルグラフ(Cf)。
0.78
section 3.4) contains many duplicate nodes that have the same feature (e g , two ENTRY nodes will be
セクション3.4) 同じ特徴を持つ重複ノード(例:2つのENTRYノード)が多数含まれている。
0.72
英語（論文から抽出）
日本語訳
スコア
Universal Representation for Code 7
コードの普遍表現 7
0.77
induced from two methods).
2つの方法から導かれる)。
0.59
These duplicate nodes serve as anchors to imply underlying relations among different graphs.
これらの重複ノードは、異なるグラフ間の基礎的な関係を示すアンカーとして機能する。
0.52
We divide duplicate nodes into two categories: strict equality and weak equality.
重複ノードを厳密な平等と弱い平等の2つのカテゴリに分けます。
0.70
Strict equality refers to duplicate nodes whose semantic meaning should be invariant to their context, including keywords (if, while, do ...), operators (=, *, << ...), entry and exit nodes.
Strict equalityは、キーワード(if, while, do ...)、演算子(=, *, << ...)、エントリと終了ノードを含む、意味的意味が文脈に不変であるべき重複ノードを指す。
0.74
Duplicate nodes of strict equality will have the same embedding in all σ graphs.
厳密な等式を持つ重複ノードは、すべてのσグラフに同じ埋め込みを持つ。
0.71
We keep a global embedding matrix to maintain their embeddings.
埋め込みを維持するために、グローバル組み込みマトリックスを保持します。
0.61
Weak equality refers to other duplicate nodes whose semantic meaning can be affected by their context.
弱い平等とは、意味的意味が文脈によって影響を受ける他の重複ノードを指す。
0.63
For example, two foo() nodes in two methods, or two tied nodes due to the simple qualified types of one or two nodes1.
We use NT signal to encourage duplicate node of weak equality to have similar embeddings: 2}, v has feature k),
NT信号を使用して、弱い等価性の重複ノードに類似の埋め込みを奨励する: 2}, v は特徴 k を持つ)。
0.76
(4) where K is the set of distinct node features (exclude strict equality nodes), ave(·) is an average function, and gk = ave({hv}, v has label k) is a global summary of nodes featured k. In (4), we first group nodes featured k, followed by computing the group centers gk, then minimize the distance between nodes to their group centers.
gk = ave({hv}, v has label k) は k を特徴とするノードのグローバル要約である。(4) (4) では、最初のグループノードが k を特徴付け、次にグループセンター gk を演算し、次にノードとグループセンターの間の距離を最小化する。 訳抜け防止モード: (4 ) ここで K は異なるノード特徴の集合である(厳密な等式ノードを除く)。 ave ( · ) は平均関数であり、gk = ave({hv }) である。 v has label k ) is a global summary of node featured k. In ( 4 ) 最初のグループノードはkで、次にグループセンターgkで計算します。 ノードからグループセンターまでの距離を 最小にします
0.87
Pre-training objective. We combine the four pre-training signals to yield a final objective:
事前訓練目的。 4つの事前トレーニング信号を組み合わせて最終目標を作ります。
0.68
(cid:88) k∈K
(cid:88) k∈K
0.65
LN T = ave({||hv − gk||2
LN T = ave({||hv − gk||2
0.77
(5) where ω1, .
(5) ここで ω1, 。
0.77
. . , ω4 balance the importance of different signals.
. . ω4は異なる信号の重要性のバランスをとる。
0.81
The objective resembles the objective in multi-task learning [19].
目的はマルチタスク学習の目的に似ています [19]。
0.80
L = ω1LM RW + ω2LHIM + ω3LM T + ω4LN T ,
L = ω1LM RW + ω2LHIM + ω3LM T + ω4LN T ,
0.75
4.2 Data Pre-processing and Fine-tuning Numeric node features.
4.2 データ前処理と微調整の数値ノード機能。
0.65
The initial node features are strings (Cf.
最初のノード機能は文字列(Cf)です。
0.75
section 3.3), which need to be cast into numeric forms before feeding into the GNN encoder.
セクション 3.3) は、gnnエンコーダに入力する前に数値形式にキャストする必要がある。
0.75
To this end, we first split each node’s feature into subtokens based on the delimiter “.”.
そのためには、まず各ノードの機能を区切り「.」に基づいてサブトークンに分割します。
0.76
Then, language models are used to get subtoken embeddings, in which we use FastText [20].
次に、言語モデルを使用して、fasttext [20]を使用するサブトケン埋め込みを取得します。
0.71
Finally, we use average subtoken embeddings as the node’s numeric feature.
最後に、ノードの数値機能として平均サブトークン埋め込みを使用します。
0.73
Inverse edges. We enrich our σ graphs with inverse edges.
逆エッジ。 我々 は逆エッジで σ グラフを豊かにします。
0.63
Recent work has proven improved performance by adding inverse edges to ASTs [6].
最近の研究はASTに逆エッジを追加することで性能を向上した[6]。
0.72
Fine-tuning. After pre-training, we can fine-tune on downstream tasks.
微調整。 事前トレーニング後、下流のタスクを微調整できる。
0.51
Finetuning involves adding downstream classifiers on top of the pre-trained node embeddings, and predicting downstream labels.
Fig. 4. Visualization of node-level embeddings with t-SNE.
フィギュア。 4. t-SNEによるノードレベルの埋め込みの可視化
0.63
Fig. 5. Visualization of method-level embeddings with t-SNE.
フィギュア。 5. t-SNEによるメソッドレベルの埋め込みの可視化
0.62
Colored with K-means. In Fig 4, we see that data nodes and action nodes are forming separate clusters, indicating our code embeddings preserve important node type information.
Fig 5 suggests method embeddings are forming discriminative clusters.
fig 5は、埋め込みメソッドが差別クラスタを形成することを示唆している。
0.51
By manually annotating each cluster, we discovered that cluster 4 contains 91% (out of all) set functions, cluster 3 contains 78% find functions, and cluster 1 contains 69% functions which end with Exception.
As in [6], we use independent linear classifiers to predict each subtoken.
6]と同様、独立線形分類器を用いて各サブトークンを予測する。
0.75
The task is defined on the method-level: predict one name for one method (code graph).
タスクはメソッドレベルで定義されます:1つのメソッド(コードグラフ)の名前を1つ予測します。
0.69
We use attention pooling [24] to generate a single embedding per method.
注意プール[24]を使用して、メソッドごとに単一の埋め込みを生成します。
0.56
We follow [6,16] to report F1, precision, recall for evaluation.
F1、精度、評価のためのリコールを報告するために[6,16]に従う。
0.54
Below we show results on Java and Python separately, as they are used for different testing purposes.
以下は、異なるテスト目的で使用されるため、JavaとPythonで個別に結果を示す。
0.78
Java. We evaluate the performance gain achieved by switching from the σ0 graph to the σ1 graph.
Java。 σ0グラフからσ1グラフに切り替えることで得られた性能向上を評価する。
0.81
We truncate subtoken sequences to a maximal length of 5 to cover 95% of the method names.
メソッド名の95%をカバーするために、サブトケン配列を最大長さ5に切り詰める。
0.77
Vocabulary size is set to 1,000, covering 95% of tokens.
語彙のサイズは1000に設定され、トークンの95%をカバーする。
0.67
Tokens not in the vocabulary are replaced by a special unknown token.
語彙にないトークンは、特別な未知のトークンに置き換えられる。
0.68
Similar techniques have been adopted in [6].
同様の技法が[6]で採用されている.
0.75
We experiment on approximately 774,000 methods from randomly selected 1,000 packages.
ランダムに選択された1,000パッケージから約774,000メソッドを実験する。
0.58
The result is summarized in Table 2.
結果は表2にまとめられている。
0.84
We see that the σ1 graph outperforms the σ0 graph, indicating that the extra information provided by the σ1 graph is beneficial for abstracting code snippets.
Fig. 7. Examples of method name prediction on Python in different degree of consensus.
フィギュア。 7. コンセンサスの程度が異なるPythonにおけるメソッド名予測の例。
0.68
Each pair of results is demonstrated as ground truth name and predicted name.
それぞれの結果は、接地真理名と予測名として示される。
0.75
try: def wait_for_task_ended( self): waiter = self.client.\ get_waiter('job_execution_comple te') # timeout is managed by airflow waiter.config.max_at tempts = sys.maxsize waiter.wait(jobs=[self.jobId]) # If waiter not available use expo retry ...
The results suggest that node embeddings from UniCoRN capture neighborhood correlations.
結果は,UniCoRNのノード埋め込みが近傍相関を捉えることを示唆している。
0.59
We see σ1 graph again improves σ0 graph.
σ1グラフは再びσ0グラフを改善する。
0.79
Fig 9 demonstrates the histogram of scores for 1,000 positive and 1,000 negative edges.
fig 9は1000の正と1000の負の辺のスコアのヒストグラムを示している。
0.68
The score, which ranges from 0 to 1, indicates the plausibility of the link existence.
0から1までの範囲のスコアは、リンクの存在の可能性を示しています。
0.74
Positive edges (0.58±0.33) score higher than negative edges (0.13±0.15), with p-value less than 0.00001 using t-test, suggesting that UniCoRN is capable to distinguish positive and negative links in code graphs.
正のエッジ (0.58±0.33) は負のエッジ (0.13±0.15) よりも高く、p-値が0.00001未満でt-test を使用しており、UniCoRN は符号グラフの正と負のリンクを区別できることを示している。 訳抜け防止モード: 正縁(0.58±0.33 )は負縁(0.13±0.15 )よりも高い。 p - t - test を用いて 0.00001 未満の値を持つ。 UniCoRNはコードグラフの正と負のリンクを区別できる。
0.66
6 Conclusion This paper presents a new model, UniCoRN, to provide a universal representation for code.
6 結論 本論文では,コードの普遍表現を提供するための新しいモデル unicorn を提案する。
0.70
Building blocks of UniCoRN include a novel σ graph to represent code as graphs, and four effective signals to pre-train GNNs.
References 1. E. Dinella, H. Dai, Z. Li, M. Naik, L. Song, and K. Wang, “Hoppity: Learning graph transformations to detect and fix bugs in programs,” in ICLR, 2019.
参照1。 E. Dinella, H. Dai, Z. Li, M. Naik, L. Song, K. Wang 氏は ICLR, 2019 で "Hoppity: Learning graph transformations to detect and fix bugs in program" を発表した。
0.77
Fig. 9. Scores for positive and negative edges.
フィギュア。 9. 正および負の端のためのスコア。
0.64
$.470
4:39!489;0 0,9;0
$.470
4:39!489;0 0,9;0
0.77
英語（論文から抽出）
日本語訳
スコア
12 L. Liu et al
12 L. Liu et al
0.92
2. J. Cambronero, H. Li, S. Kim, K. Sen, and S. Chandra, “When deep learning met code search,” in ESEC/FSE, pp.
2. J. Cambronero, H. Li, S. Kim, K. Sen, S. Chandra, “When Deep Learning met code search” in ESEC/FSE, pp。
0.90
964–974, 2019.
964–974, 2019.
0.84
3. V. Raychev, M. Vechev, and E. Yahav, “Code completion with statistical language models,” in PLDI, pp.
3. V. Raychev, M. Vechev, E. Yahav, “Code completion with statistics language model” in PLDI, pp。
0.84
419–428, 2014.
419–428, 2014.
0.84
4. M. Allamanis, M. Brockschmidt, and M. Khademi, “Learning to represent programs with graphs,” in ICLR, 2018.
4. M. Allamanis, M. Brockschmidt, M. Khademi, “Learning to representative programs with graphs” in ICLR, 2018.
0.85
5. L. Mou, G. Li, L. Zhang, T. Wang, and Z. Jin, “Convolutional neural networks over tree structures for programming language processing,” in AAAI, 2016.
5. L. Mou, G. Li, L. Zhang, T. Wang, Z. Jin, “Convolutional Neural Network over tree structure for programming language processing” in AAAI, 2016
0.86
6. W. Hu, M. Fey, M. Zitnik, Y. Dong, H. Ren, B. Liu, M. Catasta, and J. Leskovec, “Open graph benchmark: Datasets for machine learning on graphs,” NeurIPS, 2020.
6. W. Hu, M. Fey, M. Zitnik, Y. Dong, H. Ren, B. Liu, M. Catasta, J. Leskovec, “Open graph benchmark: Datasets for machine learning on graphs”, NeurIPS, 2020。
0.90
7. A. Hindle, E. T. Barr, Z. Su, M. Gabel, and P. Devanbu, “On the naturalness of software,” in ICSE, pp.
7. A. Hindle, E. T. Barr, Z. Su, M. Gabel, P. Devanbu, “On the naturalness of software” in ICSE, pp。
0.88
837–847, IEEE, 2012.
837-847, IEEE, 2012。
0.84
8. V. N. Ioannidis, D. Zheng, and G. Karypis, “Panrep: Universal node embeddings for heterogeneous graphs,” DLG-KDD, 2020.
8. V. N. Ioannidis, D. Zheng, G. Karypis, “Panrep: Universal node embeddeds for heterogeneous graphs”, DLG-KDD, 2020。
0.88
9. Z. Wu, S. Pan, F. Chen, G. Long, C. Zhang, and S. Y. Philip, “A comprehensive survey on graph neural networks,” TNNLS, 2020.
9. Z.Wu, S. Pan, F. Chen, G. Long, C. Zhang, S. Y. Philip, “A comprehensive survey on graph neural network”, TNNLS, 2020。
0.88
10. M. Schlichtkrull, T. N. Kipf, P. Bloem, R. Van Den Berg, I. Titov, and M. Welling, “Modeling relational data with graph convolutional networks,” in ESWC, pp.
10. M. Schlichtkrull, T. N. Kipf, P. Bloem, R. Van Den Berg, I. Titov, M. Welling, “Modeling relational data with graph convolutional network” in ESWC, pp。
0.89
593– 607, Springer, 2018.
593–607, Springer, 2018年。
0.87
11. W. Hu, B. Liu, J. Gomes, M. Zitnik, P. Liang, V. Pande, and J. Leskovec, “Strategies for pre-training graph neural networks,” ICLR, 2020.
11. W. Hu, B. Liu, J. Gomes, M. Zitnik, P. Liang, V. Pande, J. Leskovec, “Strategies for pre-training graph neural network”, ICLR, 2020。
0.90
12. W. Jin, T. Derr, H. Liu, Y. Wang, S. Wang, Z. Liu, and J. Tang, “Self-supervised learning on graphs: Deep insights and new direction,” arXiv preprint, 2020.
12. W. Jin, T. Derr, H. Liu, Y. Wang, S. Wang, Z. Liu, J. Tang, “Self-supervised learning on graphs: Deep insights and new direction”, arXiv preprint, 2020。
0.91
13. A. Kanade, P. Maniatis, G. Balakrishnan, and K. Shi, “Learning and evaluating contextual embedding of source code,” ICML, 2020.
13. A. Kanade, P. Maniatis, G. Balakrishnan, K. Shi, “Learning and Evaluation contextual embeddedding of source code” ICML, 2020。
0.86
14. Z. Feng, D. Guo, D. Tang, N. Duan, X. Feng, M. Gong, L. Shou, B. Qin, T. Liu, D. Jiang, et al , “Codebert: A pre-trained model for programming and natural languages,” EMNLP, 2020.
14. Z. Feng, D. Guo, D. Tang, N. Duan, X. Feng, M. Gong, L. Shou, B. Qin, T. Liu, D. Jiang, et al , “Codebert: A pre-trained model for programming and natural languages”, EMNLP, 2020。
0.90
15. A. Svyatkovskiy, S. K. Deng, S. Fu, and N. Sundaresan, “Intellicode compose: Code generation using transformer,” ESEC/FSE, 2020.
15. A. Svyatkovskiy, S. K. Deng, S. Fu, N. Sundaresan, “Intellicode compose: Code generation using transformer”, ESEC/FSE, 2020。
0.91
16. U. Alon, S. Brody, O.
16. Alon, S. Brody, O。
0.81
Levy, and E. Yahav, “code2seq: Generating sequences from structured representations of code,” ICLR, 2019.
Levy, and E. Yahav, “code2seq: Generating sequences from structured representations of code”, ICLR, 2019。
0.87
17. J. Ferrante, K. J. Ottenstein, and J. D. Warren, “The program dependence graph and its use in optimization,” TOPLAS, 1987.
17. J. Ferrante, K. J. Ottenstein, J. D. Warren, “The program dependent graph and its use in optimization”, TOPLAS, 1987。
0.87
18. N. K. Ahmed, J. Neville, R. A. Rossi, N. G. Duffield, and T. L. Willke, “Graphlet decomposition: Framework, algorithms, and applications,” KAIS, vol.
18. N. K. Ahmed, J. Neville, R. A. Rossi, N. G. Duffield, T. L. Willke, “Graphlet decomposition: Framework, Algorithm, and Applications”, KAIS, vol。
0.91
50, no. 3, pp. 689–722, 2017.
50だ 3、p。 689–722, 2017.
0.63
19. Y. Zhang and Q. Yang, “A survey on multi-task learning,” CoRR, 2017.
20. P. Bojanowski, E. Grave, A. Joulin, and T. Mikolov, “Enriching word vectors with subword information,” TACL, vol.
20. P. Bojanowski, E. Grave, A. Joulin, T. Mikolov, “Enriching word vectors with subword information”, TACL, vol。
0.86
5, pp. 135–146, 2017.
5, pp。 135–146, 2017.
0.82
21. D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” ICLR, 2015.
21. D.P. Kingma, J. Ba, “Adam: A method for stochastic optimization”, ICLR, 2015
0.83
22. M. Wang, D. Zheng, Z. Ye, Q. Gan, M. Li, X.
22. M. Wang, D. Zheng, Z. Ye, Q. Gan, M. Li, X。
0.88
Song, J. Zhou, C. Ma, L. Yu, Y. Gai, T. Xiao, T. He, G. Karypis, J. Li, and Z. Zhang, “Deep graph library: A graph-centric, highly-performant package for graph neural networks,” arXiv, 2019.
Song, J. Zhou, C. Ma, L. Yu, Y. Gai, T. Xiao, T. He, G. Karypis, J. Li, Z. Zhang, “ディープグラフライブラリ:グラフ中心で、グラフニューラルネットワークの高性能なパッケージ。
0.81
23. L. v. d. Maaten and G. Hinton, “Visualizing data using t-sne,” JMLR, vol.
23. L. v. d. Maaten, G. Hinton, “t-sneでデータを視覚化する”, JMLR, vol。
0.78
9, no. Nov, pp.
9、いいえ。 11月、p。
0.67
2579–2605, 2008.
2579–2605, 2008.
0.84
24. Y. Li, D. Tarlow, M. Brockschmidt, and R. Zemel, “Gated graph sequence neural networks,” ICLR, 2016.
24. Y.Li, D. Tarlow, M. Brockschmidt, R. Zemel, “Gated graph sequence neural Network” ICLR, 2016年。
0.87
25. B. Yang, W.-t. Yih, X.
25. B. Yang, W.-t. Yih, X。
0.82
He, J. Gao, and L. Deng, “Embedding entities and relations for learning and inference in knowledge bases,” ICLR, 2015.
he, j. gao, l. deng, “embedding entities and relations for learning and inference in knowledge bases”, iclr, 2015” (英語)