An emerging direction of quantum computing is to establish meaningful quantum
applications in various fields of artificial intelligence, including natural
language processing (NLP). Although some efforts based on syntactic analysis
have opened the door to research in Quantum NLP (QNLP), limitations such as
heavy syntactic preprocessing and syntax-dependent network architecture make
them impracticable on larger and real-world data sets. In this paper, we
propose a new simple network architecture, called the quantum self-attention
neural network (QSANN), which can make up for these limitations. Specifically,
we introduce the self-attention mechanism into quantum neural networks and then
utilize a Gaussian projected quantum self-attention serving as a sensible
quantum version of self-attention. As a result, QSANN is effective and scalable
on larger data sets and has the desirable property of being implementable on
near-term quantum devices. In particular, our QSANN outperforms the best
existing QNLP model based on syntactic analysis as well as a simple classical
self-attention neural network in numerical experiments of text classification
tasks on public data sets. We further show that our method exhibits robustness
to low-level quantum noises.
1Institute for Quantum Computing, Baidu Research, Beijing 100193, China
1Institute for Quantum Computing, Baidu Research, Beijing 100193, China
0.46
2Centre for Quantum Software and Information, University of Technology Sydney, NSW
2Centre for Quantum Software and Information, University of Technology Sydney, NSW
0.46
2007, Australia
2007年オーストラリア
0.77
Abstract An emerging direction of quantum computing is to establish meaningful quantum applications in various fields of artificial intelligence, including natural language processing (NLP).
Although some efforts based on syntactic analysis have opened the door to research in Quantum NLP (QNLP), limitations such as heavy syntactic preprocessing and syntax-dependent network architecture make them impracticable on larger and real-world data sets.
In this paper, we propose a new simple network architecture, called the quantum self-attention neural network (QSANN), which can make up for these limitations.
Specifically, we introduce the self-attention mechanism into quantum neural networks and then utilize a Gaussian projected quantum self-attention serving as a sensible quantum version of self-attention.
In particular, our QSANN outperforms the best existing QNLP model based on syntactic analysis as well as a simple classical self-attention neural network in numerical experiments of text classification tasks on public data sets.
We further show that our method exhibits robustness to low-level quantum noises.
さらに,本手法は低レベル量子雑音に対するロバスト性を示す。
0.69
1 Introduction Quantum computing is a promising paradigm [1] for fast computations that can provide substantial advantages in solving valuable problems [2, 3, 4, 5, 6].
With major academic and industry efforts on developing quantum algorithms and quantum hardware, it has led to an increasing number of powerful applications in areas including optimization [7], cryptography [8], chemistry [9, 10], and machine learning [6, 11, 12, 13].
But such devices with 50-100 qubits already allow one to achieve quantum advantage against the most powerful classical supercomputers on certain carefully designed tasks [15, 16].
To explore practical applications with near-term quantum devices, plenty of NISQ algorithms [17, 18, 19] appear to be the best hope for obtaining quantum advantage in fields such as quantum chemistry [20], optimization [21], and machine learning [22, 23, 24].
In particular, those algorithms dealing with machine learning
特に機械学習を扱うアルゴリズムは
0.61
1
1
0.42
英語（論文から抽出）
日本語訳
スコア
problems, by employing parameterized quantum circuits (PQCs) [25] (also called quantum neural networks (QNNs) [26]), show great potential in the field of quantum machine learning (QML).
Another approach, known as quantum natural language processing (QNLP), seeks to develop quantum-native NLP models that can be implemented on quantum devices [31, 32, 33, 34].
Most of these QNLP proposals, though at the frontier, lack scalability as they are based on syntactic analysis, which is a preprocessing task requiring significant effort, especially for large data sets.
Furthermore, these syntax-based methods employ different PQCs for sentences with different syntactical structures and thus are not flexible enough to process the innumerable complex expressions possible in human language.
To overcome these drawbacks in current QNLP models, we propose the quantum self-attention neural network (QSANN), where the self-attention mechanism is introduced into quantum neural networks.
Our motivation comes from the excellent performance of self-attention on various NLP tasks such as language modeling [35], machine translation [36], question answering [37], and text classification [38].
We also note that a recently proposed method [39] for quantum state tomography, an important task in quantum computing, adopts the self-attention mechanism and achieves decent results.
In each quantum self-attention layer of QSANN, we first encode the inputs into high-dimensional quantum states, then apply PQCs on them according to the layout of the self-attention neural networks, and finally adopt a Gaussian projected quantum self-attention (GPQSA) to obtain the output effectively.
To evaluate the performance of our model, we conduct numerical experiments of text classification with different data sets.
モデルの性能を評価するため,異なるデータセットを用いたテキスト分類の数値実験を行った。
0.88
The results show that QSANN outperforms the currently best known QNLP model as well as a simple classical self-attention neural network on test accuracy, implying potential quantum advantages of our method.
• Our proposal is the first QNLP algorithm with a detailed circuit implementation scheme based on the self-attention mechanism.
•提案手法は,自己認識機構に基づく詳細な回路実装方式を用いた最初のQNLPアルゴリズムである。
0.78
This method can be implemented on NISQ devices and is more practicable on large data sets compared with previously known QNLP methods based on syntactic analysis.
• In QSANN, we introduce the Gaussian projected quantum self-attention, which can efficiently dig out the correlations between words in high-dimensional quantum feature space.
• We experimentally demonstrate that QSANN outperforms existing QNLP methods based on syntactic analysis [40] and simple classical self-attention neural networks on several public data sets for text classification.
In particular, a pure quantum state could be represented by a unit vector |ψ(cid:105) ∈ C2n (or (cid:104)ψ|), where the ket notation |(cid:105) denotes a column vector and the bra notation (cid:104)ψ| = |ψ(cid:105)† with † referring to conjugate transpose denotes a row vector.
The evolution of a pure quantum state |ψ(cid:105) is mathematically described by applying a quantum circuit (or a quantum gate), i.e., |ψ(cid:48)(cid:105) = U |ψ(cid:105), where U is the unitary operator (matrix) representing the quantum circuit and |ψ(cid:48)(cid:105) is the quantum state after evolution.
Common single-qubit quantum gates include Hadamard gate H and Pauli operators
一般的な単一量子ビット量子ゲートには、ハダマールゲート h とパウリ作用素がある。
0.50
(cid:20)1 1
(出典:20)1-1
0.53
(cid:21) 1√ 2
(出典:21) 1√ 2
0.56
(cid:20)0 1 (cid:21)
(cid:20)01(cid:21)
0.42
1 0 (cid:20)0 −i (cid:21)
1 0 (cid:20)0 −i(cid:21)
0.40
i 0 (cid:20)1 0
私は0 (cid:20)1 0
0.59
(cid:21) H :=
(出典:21) H :=
0.55
, Y := , X :=
, Y := , X :=
0.43
1 −1 0 −1 and their corresponding rotation gates denoted by RP (θ) := exp(−iθP/2) = cos θ 2 P , where the rotation angle θ ∈ [0, 2π) and P ∈ {X, Y, Z}.
1 −1 0 −1 と対応する回転ゲートは rp (θ) := exp(−iθp/2) = cos θ 2 p で表され、回転角 θ ∈ [0, 2π) と p ∈ {x, y, z} が表される。
0.67
Multiple-qubit quantum gates mainly include, in this paper, the identity gate I, the CNOT gate and the tensor product of single-qubit gates, e g , Z ⊗ Z, Z ⊗ I, Z⊗n and so on.
多重量子ゲート(multiple-qubit quantum gate)は、主に、単位ゲート i, cnotゲート、シングルキュービットゲートのテンソル積、eg,z,z,i,z,nなどを含む。
0.64
Quantum measurement is a way to extract classical information from a quantum state.
量子計測は、量子状態から古典的情報を抽出する方法である。
0.84
For instance, given a quantum state |ψ(cid:105) and an observable O, one could design quantum measurements to obtain the information (cid:104)ψ| O |ψ(cid:105).
例えば、量子状態 | (cid:105) と観測可能な O が与えられたとき、その情報を得るために量子測度を設計することができた(cid:104)。
0.82
Within this work, we focus on the hardware-efficient Pauli measurements, i.e., setting O as Pauli operators or their tensor products.
1.1.3 Self-Attention Mechanism In a self-attention neural network layer [36], the input data {xs ∈ Rd}S s=1 are linearly mapped, via three weight matrices, i.e., query Wq ∈ Rd×d, key Wk ∈ Rd×d and value Wv ∈ Rd×d, to three parts Wqxs, Wkxs, Wvxs, respectively, and by applying inner product on the query and key parts, the output is computed as
with as,j = (cid:80)S ex(cid:62) s W (cid:62) l=1 ex(cid:62)
と as,j = (cid:80)S ex(cid:62) s W (cid:62) l=1 ex(cid:62)
0.47
q Wkxj s W (cid:62)
q Wkxj s W (cid:62)
0.47
q Wkxl , (2)
q Wkxl , (2)
0.43
where as,j denote the self-attention coefficients.
ここで、jは自己注意係数を表します。
0.48
j=1 3
j=1 3
0.36
英語（論文から抽出）
日本語訳
スコア
Figure 1: Sketch of a quantum self-attention layer (QSAL).
図1:qsal(quantum self-attention layer)のスケッチ。
0.65
On quantum devices, the classical inputs {y(l−1) } are used as the rotation angles of quantum ansatzes (purple dashed boxes) to encode them into their corresponding quantum states {|ψs(cid:105)}.
Then for each state, there are three different classes of ansatzes (red dashed boxes) need to be executed, where the top two classes denote the query and key parts, and the bottom one denotes the value part.
On classical computers, the measurement outputs of the query part (cid:104)Zq(cid:105) s and the key part (cid:104)Zk(cid:105) j are computed through a Gaussian function to obtain the quantum self-attention coefficients αs,j (green circles); we calculate classically weighted sums of the measurement outputs of the value part (small colored squares) and add the inputs to get the outputs {y(l)
s }, where the weights are the normalized coefficients ˜αs,j, cf.
s } ここで、重みは正規化係数 sαs,j,cf である。
0.72
Eq (7).
eq (7) である。
0.78
s 2 Method In this section, we will introduce the QSANN in detail, which mainly consists of quantum selfattention layer (QSAL), loss function, analytical gradients and analysis.
s 2 方法 本稿では、量子自己保持層(QSAL)、損失関数、解析勾配、分析からなるQSANNについて詳しく紹介する。
0.46
2.1 Quantum Self-Attention Layer In the classical self-attention mechanism [36], there are mainly three components (vectors), i.e., queries, keys and values, where queries and keys are computed as weights assigned to correspondInspired by this mechanism, in QSAL we design the quantum ing values to obtain final outputs.
Concretely, for each input state |ψs(cid:105), we denote by (cid:104)Zq(cid:105) s and (cid:104)Zk(cid:105) s the Pauli-Z1 measurement outputs of the query and key parts, respectively, where
具体的には、各入力状態 |ψs(cid:105) について (cid:104)zq(cid:105) s と (cid:104)zk(cid:105) で表され、それぞれクエリとキー部分の pauli-z1 測定出力を示す。
0.70
The measurement outputs of the value part are represented by a d-dimensional vector
(4) q (θq)z1uq(θq)|ψs(cid:105) , (cid:104)zq(cid:105) s := (cid:104)ψs| u ~ k(θk)z1uk(θk)|ψs(cid:105) である。
0.56
(cid:104)Zk(cid:105) s := (cid:104)ψs| U (cid:3)(cid:62)
(cid:104)Zk(cid:105) s := (cid:104)*s| U (cid:3)(cid:62)
0.38
os :=(cid:2)(cid:104)P1(c id:105)s
os :=(cid:2)(cid:104)P1(c id:105)s
0.40
(cid:104)P2(cid:105) s
(cid:104)P2(cid:105)
0.40
(cid:104)Pd(cid:105) s
(cid:104)Pd(cid:105)
0.44
··· where (cid:104)Pj(cid:105) s = (cid:104)ψs| U able.
··· ここで (cid:104)pj(cid:105) s = (cid:104)ψs| u である。
0.48
(5) † v (θv)PjUv(θv)|ψs(cid:105).
(5) v (θv)pjuv(θv)|ψs(cid:105) である。
0.70
Here, each Pj ∈ {I, X, Y, Z}⊗n denotes a Pauli observs ∈ Rd} of the l-th QSAL are
ここで、各 pj ∈ {i, x, y, z},n は l 番目の qsal の pauli observs ∈ rd} を表す。
0.76
Finally, by combining Eqs.
最後に、Eqsを組み合わせる。
0.79
(4) and (5), the classical output {y(l)
(4) と (5) は古典的な出力 {y(l) である
0.91
, computed as follows:
, 以下に計算する。
0.54
s = y(l−1) y(l)
s = y(l−1) y(l)
0.50
s + S(cid:88)
s + S(第88回)
0.49
j=1 ˜αs,j · oj,
j=1 は、αs、j · oj。
0.47
(6) where ˜αs,j denotes the normalized quantum self-attention coefficient between the s-th and the j-th input vectors and is calculated by the corresponding query and key parts:
When designing a quantum version of self-attention, a natural and direct extension of the inner-product † self-attention to consider is αs,j := |(cid:104)ψs| U q Uk |ψj(cid:105)|2.
自己アテンションの量子版を設計するとき、内積の自然な直接拡張は αs,j := |(cid:104)ψs| u q uk |ψj(cid:105)|2 である。
0.75
However, due to the unitary nature of quantum † q Uk can be regarded as rotating |ψs(cid:105) by an angle, which makes it difficult for |ψs(cid:105) to circuits, (cid:104)ψs| U simultaneously correlate those |ψj(cid:105) that are far away.
しかし、量子式 q uk のユニタリ性により、回転 |ψs(cid:105) を角度で考えることができ、 |ψs(cid:105) を回路に関連付けることは難しく、 (cid:104)ψs| u は遠く離れた |ψj(cid:105) を同時に関連付ける。
0.70
In a word, this direct extension is not suitable or reasonable for working as the quantum self-attention.
言い換えれば、この直接拡張は量子自己アテンションとして働くのに適切あるいは妥当ではない。
0.68
Instead, the particular quantum self-attention proposed in Eq (7), which we call Gaussian projected quantum self-attention (GPQSA), could overcome the above drawback.
In GPQSA, the states Uq |ψs(cid:105) (and Uk |ψj(cid:105)) in large quantum Hilbert space are projected to classical representations (cid:104)Zq(cid:105) s (and (cid:104)Zk(cid:105) j) in one-dimensional1 classical space via quantum measurements, and a Gaussian function is applied to these classical representations.
GPQSAでは、大きな量子ヒルベルト空間における状態 Uq | s(cid:105) (および Uk | j(cid:105)) は古典的表現 (cid:104)Zq(cid:105) s (and (cid:104)Zk(cid:105) j) に量子測度を通して一次元の古典的空間において射影され、ガウス函数はこれらの古典的表現に適用される。
0.72
As Uq and Uk are separated, it’s pretty easier to correlate |ψs(cid:105) to any |ψj(cid:105), making GPQSA more suitable to serve as a quantum self-attention.
Here, we utilize the Gaussian function [42] mainly because it contains infinite-dimensional feature space and is well-studied in classical machine learning.
ガウス関数 [42] は主に無限次元の特徴空間を持ち、古典的機械学習でよく研究されている。
0.72
Numerical experiments also verify our choice of Gaussian function.
数値実験はガウス函数の選択も検証する。
0.68
We also note that other choices of building the quantum self-attention are also worth a future study.
また、量子自己注意を構築する他の選択肢も将来の研究に値することに留意する。
0.69
1Multi-dimension is also possible by choosing multiple measurement results, like the value part.
1Multi次元は、値部分のように複数の測定結果を選択することでも可能である。
0.66
5
5
0.42
英語（論文から抽出）
日本語訳
スコア
Rx(θ0,1) Ry(θ0,5)
Rx(θ0,1) Ry(θ0,5)
0.37
Rx(θ0,2) Ry(θ0,6)
Rx(θ0,2) Ry(θ0,6)
0.37
Rx(θ0,3) Ry(θ0,7)
Rx(θ0,3) Ry(θ0,7)
0.37
Rx(θ0,4) Ry(θ0,8)
Rx(θ0,4) Ry(θ0,8)
0.37
• • • Ry(θ1,1)
• • • Ry(θ1,1)
0.41
Ry(θ1,2)
ry (複数形 rys)
0.39
Ry(θ1,3) Ry(θ1,4)
Ry(θ1,3) Ry(θ1,4)
0.37
• ×D Figure 2: The ansatz used in QSANN.
• ×D 図2:QSANNで使用されるアンザッツ。
0.53
The first two columns denote the Rx-Ry rotations on each single-qubit subspace, then followed by repeated CNOT gates and single-qubit Ry rotations.
最初の2つの列は、各1キュービット部分空間上のRx-Ry回転を表し、続いて繰り返しCNOTゲートと1キュービットRy回転を示す。 訳抜け防止モード: 最初の2つの列は rx - ry 回転を表す。 その後、連続するcnotゲートとシングル-キュービットry回転が続く。
0.74
The block circuit in the dashed box is repeated D times to enhance the expressive power of the ansatz.
破断箱内のブロック回路は、アンザッツの表現力を高めるために、繰り返しDされる。
0.60
Figure 3: Sketch of QSANN, where a sequence of classical vectors {xs} firstly goes through L QSALs to s }, then through the average operation, and finally obtain the corresponding sequence of feature vectors {y(L) through the fully-connected layer for the binary prediction task.
図3: QSANN のスケッチでは、古典ベクトル {xs} の列が L QSALs から s } に進み、次に平均演算を経て、最後に二項予測タスクの完全連結層を通して特徴ベクトル {y(L) の列を得る。
0.71
Remark. During the preparation of this manuscript as well as after submitting our work to a peerreview conference in Sep 2021, we became aware that Ref.
[43] also made initial attempts to employ the attention mechanism in QNNs.
また[43]では,QNNにおける注意機構の活用も試みている。
0.70
In that work, the authors mentioned a possible quantum extension towards a quantum Transformer where the straightforward inner-product self-attention is adopted.
その研究の中で、著者らは量子変換器への量子拡張の可能性について言及した。
0.63
As discussed above, the inner-product self-attention may not be reasonable for dealing with quantum data.
上述したように、内積自己注意は量子データを扱うには妥当ではないかもしれない。
0.58
In this work, we present that GPQSA is more suitable for the quantum version of self-attention and show the validity of our method via numerical experiments on several public data sets.
The whole procedure of QSANN is depicted in Fig 3, which mainly consists of L QSALs to extract hidden features and one fully-connected layer to complete the binary prediction task.
Here, the mean squared error [44] is employed as the loss function:
ここでは損失関数として平均二乗誤差[44]を用いる。
0.74
L (Θ, w, b; D) =
L ( , w, b; D) =
0.35
1 2Ns m=1 Ns(cid:88)
1 2N m=1。 Ns(第88回)
0.49
(cid:16)(m) ˆy − (m)y
(cid:16)(m) sy − (m)y
0.45
(cid:17)2 (cid:16)
(出典:17)2 (出典:16)
0.70
+ RegTerm, (cid:80)Sm
+条例 (cid:80)sm
0.40
(cid:17) with w ∈ Rd where the predicted value (m) ˆy is defined as (m) ˆy := σ and b ∈ R denoting the weights and bias of the final fully-connected layer, Θ denoting all parameters in the ansatz, σ denoting the sigmoid activation function and ‘RegTerm’ being the regularization term to avoid overfitting in the training process.
(cid:17) w ∈ Rd では、予測値 (m) y は (m) := σ と b ∈ R と定義され、最終完全連結層の重みと偏りを表し、 σ はアンザッツ内の全てのパラメータを表し、σ はシグモイド活性化関数を表し、'RegTerm' は訓練過程における過度な適合を避ける正規化項である。
0.63
w(cid:62) · 1
w(cid:62) · 1
0.46
(m)y(L) s + b
(m)y(L) s + b である。
0.51
s=1 Sm Combining Eqs.
s=1。 Sm Eqsの組み合わせ。
0.56
(3) - (7), we know each output of QSAL is dependent on all its inputs, i.e.,
(3) - (7) QSALの各出力がすべての入力に依存することを知っています。
0.84
(8) (9) (10)
(8) (9) (10)
0.42
(m)y (l) s :=
(m)y (l)s :=
0.42
(m)y (l) s q , θ(l) θ(l)
(m)y (l)s q , θ(l) θ(l)
0.42
k , θ(l) = (m)y(l−1)
k , θ(l) = (m)y(l−1)
0.45
s + ˜α(l) s,j
s + ～α(l) s,j
0.58
(cid:16) Sm(cid:88)
(出典:16) Sm (cid:88)
0.54
j=1 (cid:17) k ;{(m)y(l−1)
j=1 (cid:17) k ;{(m)y(l−1)
0.39
}Sm i=1
※Sm i=1 である。
0.26
i v ;{(m)y(l−1) (cid:16)
私は v ;{(m)y(l−1) (cid:16)
0.51
i q , θ(l) θ(l)
私は q , θ(l) θ(l)
0.48
(cid:17) · o(l)
(出典:17)· o(l)
0.86
j (cid:16) }Sm
j (出典:16) ※Sm
0.44
i=1 v ;
i=1 である。 v ;
0.37
(m)y(l−1) θ
(m)y(l−1)θ
0.48
(l) j (cid:17)
(l) j (cid:17)
0.41
, where (m)y(0)
, ここで (m)y(0)
0.64
s = (m)xs and 1 ≤ s ≤ Sm, 1 ≤ l ≤ L. Here, the regularization term is defined as
s = (m)xs と 1 ≤ s ≤ Sm, 1 ≤ l ≤ L である。 訳抜け防止モード: s = (m)xs と 1 ≤ s ≤ Sm, 1 ≤ l ≤ L である。 正規化という用語は
0.79
RegTerm :=
regterm :=
0.31
(cid:107)w(cid:107)2 +
(cid:107)w(cid:107)2 +
0.42
λ 2d γ 2d (cid:107) (m)xs(cid:107)2,
λ 2d γ2d (cid:107) (m)xs(cid:107)2,
0.44
Sm(cid:88)
Sm (cid:88)
0.41
s=1 where λ, γ ≥ 0 are two regularization coefficients.
s=1。 ここで λ, γ ≥ 0 は二つの正規化係数である。
0.56
With the loss function defined in Eq (8), we can optimize its parameters by (stochastic) gradientdescent [45].
損失関数を Eq (8) で定義すれば、パラメータを(確率的に)勾配的[45] で最適化できる。
0.80
The analytical gradient analysis can be found in the Appendix.
解析的勾配解析はAppendixで見ることができる。
0.72
Finally, with the above preparation, we could train our QSANN to get the optimal (or sub-optimal) parameters.
See Algorithm 1 for details on the training procedure.
トレーニング手順の詳細はアルゴリズム1を参照。
0.73
We remark that if the loss converges during training or the maximum number of iterations is reached, the optimization stops.
トレーニング中に損失が収束したり、イテレーションの最大数に達すると、最適化が停止する。
0.58
2.4 Analysis of QSANN According to the definition of the Quantum Self-Attention Layer, for a sequence with S words, we need S(d + 2) Pauli measurements to obtain the d-dimensional value vectors as well as the queries and keys for all words from the quantum device.
2.4QSANNの解析 量子自己保持層の定義によれば、Sワードのシーケンスでは、量子デバイスからすべての単語に対するクエリとキーだけでなく、d次元の値ベクトルを得るためにS(d + 2) Pauli測定が必要である。 訳抜け防止モード: 2.4 量子自己注意層の定義によるQSANNの分析(一般講演) S 語を含む列に対して、量子デバイスから d-次元値ベクトルと全ての単語に対するクエリとキーを得るには、S(d + 2 ) Pauli 測定が必要である。
0.90
After that, we need to compute S2 self-attention coefficients for all S2 pairs of words on the classical computer.
その後、古典的コンピュータ上で全てのS2単語に対してS2自己注意係数を計算する必要がある。
0.73
In general, QSANN takes advantage of quantum devices’ efficiency in processing high-dimensional data while outsourcing some calculations to classical computers.
Algorithm 1 QSANN training for text classification Input: The training data set D := {((m)x1, (m)x2, .
Algorithm 1 QSANN training for text classification Input: The training data set D := {((m)x1, (m)x2, .
0.45
. . , (m)xSm), (m)y}Ns Output: The final ansatz parameters Θ∗, weight w∗, b∗ 1: Initialize the ansatz parameters Θ, weight w from Gaussian distribution N(0, 0.01) and the bias b
(m)xs to get the corresponding quantum state |ψs(cid:105), cf.
(m)xs は対応する量子状態 | s(cid:105), cf を得る。
0.86
(3). Apply Uq and Uk to |ψs(cid:105) and measure the Pauli-Z expectations to get (cid:104)Zq(cid:105) s,(cid:104)Zk(cid:10 5)s, cf.
(3). Uq と Uk を | s (cid:105) に適用し、 (cid:104)Zq(cid:105) s, (cid:104)Zk(cid:105) s, cf とする。
0.56
(4), and then calculate the quantum self-attention coefficients αs,j, cf.
(4) を計算し、量子自己アテンション係数 αs,j,cf を計算する。
0.80
(7). Apply Uv and measure a series of Pauli expectations to get os, cf.
(7). uv を適用して,os と cf を取得するための pauli の一連の期待値を測定する。
0.47
(5), and then compute the output {y
(5) を計算し、次に出力 {y
0.85
(l) Repeat 4-6 L times to get the output {y(L) Average {y(L) Calculate the mean squared error in (8) and update the parameters through the optimization procedure.
(l)出力 {y(L) 平均 {y(L) 平均 {y(L) 平均二乗誤差を (8) で計算し、最適化手順でパラメータを更新する。 訳抜け防止モード: (l) Repeat 4 - 6 L times to get the output { y(L ) Average { y(L ) Calculate the mean squared error in (8) パラメータを最適化手順で更新する。
0.83
s } and through a fully-connected layer to obtain the predicted value (m) ˆy.
s } と完全に接続された層を通して予測値 (m) y を得る。
0.76
s } of the l-th QSAL, cf.
s } のl-th QSAL, cf。
0.38
(6). s } of the L-th QSAL.
(6). L-th QSAL の s } である。
0.57
5: 6: 7: 8: 9:
5: 6: 7: 8: 9:
0.42
end for if the stopping condition is met then
停止条件が満たされた場合の終了
0.67
10: 11: 12: end if 13: 14: end for
10: 11: 12: end if 13: 14: end for
0.42
Break. In short, our QSANN first encodes words into a large quantum Hilbert space as the feature space and then projects them back to low-dimensional classical feature space by quantum measurement.
Recent works have proved rigorous quantum advantages on some classification tasks by utilizing high-dimensional quantum feature space [46] and projected quantum models [47].
Thus, we expect that our QSANN might also have the potential advantage of digging out some hidden features that are classically intractable.
したがって、当社のQSANNは、古典的に難解な隠れた特徴を掘り下げる潜在的な利点を期待しています。
0.74
In the following section, we carry out numerical simulations of QSANN on several data sets to evaluate its performance on binary text classification tasks.
3 Numerical Results In order to demonstrate the performance of our proposed QSANN, we have conducted numerical experiments on public data sets, where the quantum part was accomplished via classical simulation.
Concretely, we first exhibit the better performance of QSANN by comparing it with
具体的には、まずQSANNのパフォーマンスを比較して比較する。
0.73
i) the syntactic analysis-based quantum model [40] on two simple tasks, i.e., MC and RP,
i) 2つの単純なタスク,すなわちmcとrpの構文解析に基づく量子モデル[40]
0.83
ii) the classical self-attention neural network (CSANN) and the naive method on three public sentiment analysis data sets, i.e., Yelp, IMDb and Amazon [48].
loop are implemented via Paddle Quantum2 on the PaddlePaddle Deep Learning Platform [49].
ループはPaddle Quantum2経由でPaddlePaddle Deep Learning Platform [49]上に実装されます。
0.79
3.1 Data Sets The two simple synthetic data sets we employed come directly from [40], which are named as MC and RP, respectively.
3.1 データセット われわれが採用した2つの単純な合成データセットは、それぞれ mc と rp という名前の [40] から来ている。
0.72
MC contains 17 words and 130 sentences (70 train + 30 development + 30 test) with 3 or 4 words each; RP has 115 words and 105 sentences (74 train + 31 test) with 4 words in each one.
And each sequence contains several to dozens of words.
各シーケンスには数から数十の単語が含まれています
0.61
We randomly select 80% as training sequences and the rest 20% as test ones.
トレーニングシーケンスとして80%,テストシーケンスとして残りの20%をランダムに選択します。
0.69
3.2 Experimental Setting In the experiments, we use a single self-attention layer for both QSANN and CSANN.
3.2 実験設定 実験では,QSANN と CSANN の両方に単一自己注意層を用いる。
0.86
As a comparison, we also perform the most straightforward method, i.e., averaging directly the embedded vectors of a sequence, followed by a fully-connected layer, which we call the ‘Naive’ method, on the three data sets of reviews.
In QSANN, all the encoder, query, key and value ansatzes have the same qubit number and are constructed according to Fig 2, which are easily imiplementable on the NISQ devices.
Specifically, assuming the n-qubit encoder ansatz has Denc layers with n(Denc + 2) parameters, we could just set the dimension of the input vectors as d = n(Denc + 2).
For example, it is just required 3n observables when Denc = 1.
例えば、denc = 1 の場合、3n の可観測性が必要である。
0.65
However, if Denc > 1, we could also choose two-qubit observables Z12, Z23 and so on.
しかし、denc > 1 の場合、2量子可観測性 z12, z23 などを選択することもできる。
0.72
All the ansatz parameters Θ and weight w are initialized from a Gaussian distribution with zero mean and 0.01 standard deviation, and the bias b is initialized to zero.
アンサッツパラメータ θ とウェイト w は、平均 0 と 0.01 の標準偏差を持つガウス分布から初期化され、バイアス b は 0 に初期化される。
0.83
Here, the ansatz parameters are not initialized uniformly from [0, 2π) is mainly due to the residual
During the optimization iteration, we use Adam optimizer [51].
最適化イテレーションでは、Adam Optimizationr [51]を使用します。
0.79
And we repeat each experiment 9 times with different parameter initializations to collect the average accuracy and the corresponding fluctuations.
そして,各実験を9回繰り返してパラメータ初期化を行い,平均精度と対応するゆらぎを収集した。
0.80
In CSANN, we set d = 16 and the classical query, key and value matrices are also initialized from a Gaussian distribution with zero mean and 0.01 standard deviation.
Except these, almost all other parameters are set the same as QSANN.
これらを除いて、ほとんどの他のパラメータはQSANNと同じである。
0.81
These settings and initializations are the same in the naive method as well.
これらの設定と初期化は、naiveメソッドでも同じです。
0.80
3.3 Results on MC and RP Tasks The results on MC and RP tasks are summarized in Table 2.
3.3 MCおよびRPタスクの結果 MCおよびRPタスクの結果は表2にまとめられている。
0.86
In the MC task, our method QSANN could easily achieve a 100% test accuracy while requiring only 25 parameters (18 in query-key-value part and 7 in fully-connected part).
However, we observe that both test accuracies are pretty low when compared with the training accuracy.
しかし, 両者の検査精度は, 訓練精度と比較してかなり低いことがわかった。
0.80
It is mainly because there is a massive bias between the training set and test set, i.e., more than half of the words in the test set have not appeared in the training one.
Hence, the test accuracy highly depends on random guessing.
したがって、テスト精度はランダムな推測に大きく依存する。
0.74
3.4 Results on Yelp, IMDb and Amazon Data Sets As there are no quantum algorithms for text classification on these three data sets before, we benchmark our QSANN with the classical self-attention neural network (CSANN).
3.4 Yelp、IMDb、Amazon Data Setsの成果 これら3つのデータセットのテキスト分類に量子アルゴリズムがないため、QSANNを古典的な自己注意ニューラルネットワーク(CSANN)でベンチマークする。
0.80
The naive method is 10
ナイーブな方法は 10
0.53
英語（論文から抽出）
日本語訳
スコア
Figure 4: Heat maps of the averaged quantum self-attention coefficients for some selected test sequences from the Yelp data set, where a deeper color indicates a higher coefficient.
Words that are more sentiment-related are generally assigned higher self-attention coefficients by our Gaussian projected quantum self-attention, implying the validity and interpretability of QSANN.
In comparison, QSANN has only 49 parameters (36 in querykey-value part and 13 in fully-connected part) on the Yelp and IMDb data sets and 61 parameters (48 in query-key-value part and 13 in fully-connected part) on the Amazon data set, improving the test accuracy by about 1% as well as saving more than 10 times the number of parameters.
Therefore, QSANN could have a potential advantage for text classification.
したがって、QSANNはテキスト分類に潜在的に有利である可能性がある。
0.62
3.5 Visualization of Self-Attention Coefficient To intuitively demonstrate the reasonableness of the Gaussian projected quantum self-attention, in Fig. 4 we visualize the averaged quantum self-attention coefficients of some selected test sequences from the Yelp data set.
Concretely, for a sequence, we calculate 1 s=1 ˜αs,j for j = 1, . . . , S and S visualize them via a heat map, where S is the number of words in this sequence and ˜αs,j is the quantum self-attention coefficient.
具体的には、ある列に対して、j = 1 に対して 1 s=1 >αs,j を計算し、S と S は熱写像を通してそれらを視覚化する。 訳抜け防止モード: 具体的には、ある列に対して 1 s=1 sα を計算する。 j for j = 1, . ., s と s はヒートマップでそれらを視覚化する。 s はこの列のワード数であり、j は量子自己-注意係数 である。
0.76
As shown in the figure, words with higher quantum self-attention coefficients are indeed those that determine the emotion of a sequence, implying the power of QSANN for capturing the most relevant words in a sequence on text classification tasks.
(cid:80)S 3.6 Noisy Experimental Results on Yelp Data Set Due to the limitations of the near-term quantum computers, we add experiments with noisy quantum circuits to demonstrate the robustness of QSANN on the Yelp data set.
As a regular way to analyze the effect of quantum noises, we add these single-qubit noisy channels in the final circuit layer to represent the whole system’s noise, which is illustrated in Fig 5(a).
√ We take the noise level p as 0.01, 0.1, 0.2 for these two noisy channels, respectively, and the box plots of test accuracies are depicted in Fig 5(b).
From the picture, we see the test accuracy of our QSANN almost does not decrease when the noise level is less than 0.1, and even when the noise level is up to 0.2, the overall test accuracy has only decreased a little, showing that QSANN is robust to these quantum noises.
Specifically, the adopted Gaussian projected quantum selfattention exploits the exponentially large quantum Hilbert space as the quantum feature space, making QSANN have the potential advantage of mining some hidden correlations between words that are difficult to dig out classically.
Numerical results show that QSANN outperforms the best-known QNLP method and a simple classical self-attention neural network for text classification on several public data sets.
Moreover, using only shallow quantum circuits and Pauli measurements, QSANN can be easily implemented on near-term quantum devices and is noise-resilient, as implied by simulation results.
We believe that this attempt to combine self-attention and quantum neural networks would
私たちは、この自己発見と量子ニューラルネットワークを組み合わせる試みは、
0.72
12
12
0.42
英語（論文から抽出）
日本語訳
スコア
open up new avenues for QNLP as well as QML.
QNLPとQMLの新しい道を開く。
0.59
As a future direction, more advanced techniques such as positional encoding and multi-head attention can be employed in quantum neural networks for generative models and other more complicated tasks.
Acknowledgements We would like to thank Prof. Sanjiang Li and Prof. Yuan Feng for helpful discussions.
覚書 サンジャン・リ教授と元文教授に有意義な議論をお願いいたします。
0.50
G. L. acknowledges the support from the Baidu-UTS AI Meets Quantum project, the China Scholarship Council (No. 201806070139), and the Australian Research Council project (Grant No: DP180100691).
G.L.はBaidu-UTS AI Meets Quantum Project、China Scholarship Council (No. 201806070139)、Australian Research Council Project (Grant No: DP180100691)の支援を認めている。
0.83
Part of this work was done when X. Z. was a research intern at Baidu Research.
この研究の一部は、X.Z.がBaidu Researchのインターンだった時に行われた。
0.72
References [1] John Preskill.
参考文献 ジョン・プレスキル(John Preskill)。
0.61
Quantum computing 40 years later.
40年後の量子コンピューティング。
0.78
arXiv preprint arXiv:2106.10522, 2021.
arXiv preprint arXiv:2106.10522, 2021
0.40
[2] Aram W Harrow and Ashley Montanaro.
アラム・W・ハローとアシュリー・モンタナロ。
0.44
Quantum computational supremacy. Nature,
量子計算超越性。 自然だ
0.50
549(7671):203–209, 2017.
549(7671):203–209, 2017.
0.44
[3] Andrew M Childs and Wim van Dam.
アンドリュー・M・チャイルドズとウィム・ヴァン・ダム。
0.47
Quantum algorithms for algebraic problems.
代数問題に対する量子アルゴリズム。
0.82
Reviews of Modern Physics, 82(1):1–52, jan 2010.
レビュー 現代物理学 82(1):1-52, jan 2010
0.67
[4] Ashley Montanaro.
アシュリー・モンタナロ(Ashley Montanaro)。
0.58
Quantum algorithms: an overview.
量子アルゴリズム:概要。
0.64
npj Quantum Information, 2(1):15023,
npj 量子情報 2(1):15023
0.69
nov 2016. [5] Andrew M. Childs, Dmitri Maslov, Yunseong Nam, Neil J. Ross, and Yuan Su.
ノヴ2016 5]Andrew M. Childs、Dmitri Maslov、Yunseong Nam、Neil J. Ross、Yuan Su。
0.41
Toward the first quantum simulation with quantum speedup.
量子スピードアップによる最初の量子シミュレーションに向けて
0.89
Proceedings of the National Academy of Sciences, 115(38):9456–9461, sep 2018.
国立科学アカデミー紀要、115(38):9456-9461, sep 2018。
0.62
[6] Jacob Biamonte, Peter Wittek, Nicola Pancotti, Patrick Rebentrost, Nathan Wiebe, and Seth
Secure quantum key distribution with realistic devices.
セキュア量子鍵 現実的な装置で配布します
0.62
Reviews of Modern Physics, 92(2):25002, 2020.
現代物理学 92(2):25002, 2020
0.60
[9] Sam McArdle, Suguru Endo, Al´an Aspuru-Guzik, Simon C. Benjamin, and Xiao Yuan.
9]Sam McArdle、Endo Suguru、Al ́an Aspuru-Guzik、Simon C. Benjamin、Xiao Yuan。
0.82
Quantum computational chemistry. Reviews of Modern Physics, 92(1):015003, mar 2020.
量子 計算化学 現代物理学、92(1):015003、2020年3月。
0.65
[10] Yudong Cao, Jonathan Romero, Jonathan P Olson, Matthias Degroote, Peter D. Johnson, M´aria Kieferov´a, Ian D. Kivlichan, Tim Menke, Borja Peropadre, Nicolas P D Sawaya, Sukin Sim, Libor Veis, and Al´an Aspuru-Guzik.
10]Yudong Cao, Jonathan Romero, Jonathan P Olson, Matthias Degroote, Peter D. Johnson, M ́aria Kieferov ́a, Ian D. Kivlichan, Tim Menke, Borja Peropadre, Nicolas P D Sawaya, Sukin Sim, Libor Veis, Al ́an Aspuru-Guzik 訳抜け防止モード: [10 ]ユドン・カオ、ジョナサン・ロメロ、ジョナサン・ポルソン Matthias Degroote, Peter D. Johnson, M ́aria Kieferov ́a, Ian D. Kivlichan Tim Menke, Borja Peropadre, Nicolas P D Sawaya, Sukin Sim Libor Veis, and Al ́an Aspuru - Guzik
0.85
Quantum Chemistry in the Age of Quantum Computing.
量子コンピューティング時代における量子化学
0.63
Chemical Reviews, 119(19):10856–10915, oct 2019.
Chemical Reviews, 119(19):10856–10915, oct 2019。
0.47
[11] Patrick Rebentrost, Masoud Mohseni, and Seth Lloyd.
11]パトリック・レベントロスト、マスード・モッセニ、セス・ロイド
0.39
Quantum support vector machine for big
大規模量子支援ベクトルマシン
0.79
data classification. Physical Review Letters, 113(3):130503, Sep 2014.
データ分類。 フィジカル・レビュー・レター、113(3):130503、sep 2014。
0.74
13
13
0.85
英語（論文から抽出）
日本語訳
スコア
[12] Hsin-Yuan Huang, Michael Broughton, Masoud Mohseni, Ryan Babbush, Sergio Boixo, Hartmut Neven, and Jarrod R. McClean.
Hsin-Yuan Huang氏、Michael Broughton氏、Masoud Mohseni氏、Ryan Babbush氏、Sergio Boixo氏、Hartmut Neven氏、Jarrod R. McClean氏。
0.74
Power of data in quantum machine learning.
量子機械学習におけるデータのパワー。
0.87
Nature Communications, 12(1):2631, dec 2021.
自然通信 12(1):2631, 12月 2021。
0.78
[13] Maria Schuld and Francesco Petruccione.
13] マリア・シュールドとフランチェスコ・ペトルクセネ
0.58
Machine Learning with Quantum Computers.
量子コンピュータによる機械学習。
0.87
2021. [14] John Preskill.
2021. 14] ジョン・プレスキル
0.49
Quantum Computing in the NISQ era and beyond.
NISQ時代以降の量子コンピューティング。
0.54
Quantum, 2:79, Aug 2018.
量子、2018年8月2:79。
0.65
[15] Frank Arute, Kunal Arya, Ryan Babbush, Dave Bacon, Joseph C Bardin, Rami Barends, Rupak Biswas, Sergio Boixo, Fernando GSL Brandao, David A Buell, et al Quantum supremacy using a programmable superconducting processor.
15] Frank Arute, Kunal Arya, Ryan Babbush, Dave Bacon, Joseph C Bardin, Rami Barends, Rupak Biswas, Sergio Boixo, Fernando GSL Brandao, David A Buell, et al Quantum supremacy using a programmable superconducting processor。 訳抜け防止モード: [15 ]フランク・アルトゥ,クナル・アリア,ライアン・バブッシュ, Dave Bacon, Joseph C Bardin, Rami Barends, Rupak Biswas, Sergio Boixo Fernando GSL Brandao, David A Buell, et al Quantum supremacy プログラム可能な超伝導プロセッサを使います
0.80
Nature, 574(7779):505–510, 2019.
自然誌 574(7779):505-510, 2019。
0.37
[16] Han-Sen Zhong, Hui Wang, Yu-Hao Deng, Ming-Cheng Chen, Li-Chao Peng, Yi-Han Luo, Jian Qin, Dian Wu, Xing Ding, Yi Hu, et al Quantum computational advantage using photons.
[16]ハンセン・ジン、ウイ・ワン、ユ・ホー・デン、ミン・チェン、リ・チャオ・ペン、イ・ハン・ルオ、ジャン・チン、ディアン・ウー、Xing Ding、Yi Hu、その他光子を用いた量子計算の優位性。 訳抜け防止モード: [16 ]漢-仙宗、湖王、湯-羽王、 明-Cheng Chen、Li-Chao Peng、Yi-Han Luo、 Jian Qin, Dian Wu, Xing Ding, Yi Hu 量子計算の優位性 光子を使って
0.77
Science, 370(6523):1460–1463, 2020.
370(6523):1460-1463、2020年。
0.64
[17] Kishor Bharti, Alba Cervera-Lierta, Thi Ha Kyaw, Tobias Haug, Sumner Alperin-Lea, Abhinav Anand, Matthias Degroote, Hermanni Heimonen, Jakob S Kottmann, Tim Menke, et al Noisy intermediate-scale quantum (nisq) algorithms.
Kishor Bharti, Alba Cervera-Lierta, Thi Ha Kyaw, Tobias Haug, Sumner Alperin-Lea, Abhinav Anand, Matthias Degroote, Hermanni Heimonen, Jakob S Kottmann, Tim Menke, et al Noisy intermediate-scale quantum (nisq) algorithm。
0.43
arXiv preprint arXiv:2101.08448, 2021.
arXiv preprint arXiv:2101.08448, 2021
0.40
[18] M. Cerezo, Andrew Arrasmith, Ryan Babbush, Simon C. Benjamin, Suguru Endo, Keisuke Fujii, Jarrod R. McClean, Kosuke Mitarai, Xiao Yuan, Lukasz Cincio, and Patrick J. Coles.
M. C. Cerezo, Andrew Arrasmith, Ryan Babbush, Simon C. Benjamin, Endo Suguru, Keisuke Fujii, Jarrod R. McClean, Kosuke Mitarai, Xiao Yuan, Lukasz Cincio, Patrick J. Coles 訳抜け防止モード: 18 ]m.セレゾ、アンドリュー・アラスミス、ライアン・バブブッシュ simon c. benjamin, suguru endo, keisuke fujii, jarrod r. mcclean, 三田井耕介、元清(xiao yuan)、シンシオ(lukasz cincio)、パトリック・j・コールズ(patrick j. coles)。
0.58
Variational quantum algorithms.
変分量子アルゴリズム。
0.71
Nature Reviews Physics, pages 1–29, aug 2021.
Nature Reviews Physics, page 1–29, aug 2021
0.45
[19] Suguru Endo, Zhenyu Cai, Simon C Benjamin, and Xiao Yuan.
[19]遠藤スグル、周遊Cai、Simon C Benjamin、Xiao Yuan。
0.28
Hybrid Quantum-Classical Algorithms and Quantum Error Mitigation.
ハイブリッド量子古典アルゴリズムと量子誤差緩和
0.74
Journal of the Physical Society of Japan, 90(3):032001, mar 2021.
日本物理学会誌 90(3):03 2001, mar 2021
0.53
[20] Alberto Peruzzo, Jarrod McClean, Peter Shadbolt, Man-Hong Yung, Xiao-Qi Zhou, Peter J. Love, Al´an Aspuru-Guzik, and Jeremy L. O’Brien.
Alberto Peruzzo, Jarrod McClean, Peter Shadbolt, Man-Hong Yung, Xiao-Qi Zhou, Peter J. Love, Al ́an Aspuru-Guzik, Jeremy L. O’Brien。 訳抜け防止モード: [20 ]Alberto Peruzzo, Jarrod McClean, Peter Shadbolt Man - Hong Yung, Xiao - Qi Zhou, Peter J. Love アル・アン・アスプル - グジク、ジェレミー・L・オブライエン。
0.76
A variational eigenvalue solver on a photonic quantum processor.
フォトニック量子プロセッサにおける変動固有値解法
0.64
Nature Communications, 5(1):4213, dec 2014.
Nature Communications, 5(1):4213, dec 2014
0.41
[21] Edward Farhi, Jeffrey Goldstone, and Sam Gutmann.
21]エドワード・ファリー、ジェフリー・ゴールドストーン、サム・グートマン
0.54
A Quantum Approximate Optimization Algorithm.
量子近似最適化 アルゴリズム。
0.53
arXiv:1411.4028, pages 1–16, Nov 2014.
arXiv:1411.4028, page 1–16, Nov 2014
0.35
[22] Vojtˇech Havl´ıˇcek, Antonio D. C´orcoles, Kristan Temme, Aram W. Harrow, Abhinav Kandala, Jerry M. Chow, and Jay M. Gambetta.
Aram W. Harrow氏、Abhinav Kandala氏、Jerry M. Chow氏、Jay M. Gambetta氏。 訳抜け防止モード: アントニオ・D・C・オルコレス (Antonio D. C'orcoles, Kristan Temme)。 Aram W. Harrow, Abhinav Kandala, Jerry M. Chow ジェイ・M・ガンベッタ。
0.67
Supervised learning with quantum-enhanced feature spaces.
量子エンハンス特徴空間を用いた教師あり学習
0.61
Nature, 567(7747):209–212, Mar 2019.
自然誌 567(7747):209-212, mar 2019。
0.72
[23] Maria Schuld, Alex Bocharov, Krysta M. Svore, and Nathan Wiebe.
マリア・シュルド(Maria Schuld)、アレックス・ボチャロフ(Alex Bocharov)、クリスタ・スヴォーレ(Krysta M. Svore)、ネイサン・ウィーベ(Nathan Wiebe)。
0.42
Circuit-centric quantum classifiers.
回路中心量子 分類器
0.35
Physical Review A, 101(3):032308, Mar 2020.
物理書評 A, 101(3):032308, Mar 2020
0.71
[24] K. Mitarai, M. Negoro, M. Kitagawa, and K. Fujii.
[24]三田井氏、根来氏、北川氏、藤井氏。
0.45
Quantum circuit learning. Physical Review
量子回路学習。 物理レビュー
0.73
A, 98(3):032309, Sep 2018.
A,98(3):032309, Sep 2018。
0.41
[25] Marcello Benedetti, Erika Lloyd, Stefan Sack, and Mattia Fiorentini.
25]マルチェロ・ベネデッティ、エリカ・ロイド、ステファン・サック、マティア・フィオレンティーニ
0.37
Parameterized quantum circuits as machine learning models.
機械学習モデルとしての量子回路のパラメータ化。
0.68
Quantum Science and Technology, 4(4):043001, Jun 2019.
Classification with Quantum Neural Networks on Near Term
量子ニューラルネットワークを用いた近距離領域の分類
0.71
Processors. arXiv:1802.06002, pages 1–21, Feb 2018.
プロセッサ。 arXiv:1802.06002, page 1–21, Feb 2018
0.53
14
14
0.42
英語（論文から抽出）
日本語訳
スコア
[27] Alessandro Sordoni, Jian-Yun Nie, and Yoshua Bengio.
27] アレッサンドロ・ソルドニ、ジャン・ユン・ニー、ヨシュア・ベンジオ
0.47
Modeling term dependencies with quantum language models for IR.
IRの量子言語モデルによる項依存のモデル化。
0.83
In Proceedings of the 36th international ACM SIGIR conference on Research and development in information retrieval - SIGIR ’13, page 653, New York, New York, USA, 2013.
第36回ACM SIGIR国際情報検索研究開発会議報告 - SIGIR'13, page 653, New York, New York, USA, 2013
0.62
ACM Press. [28] Peng Zhang, Jiabin Niu, Zhan Su, Benyou Wang, Liqun Ma, and Dawei Song.
Bert: Pre-training of deep bidirectional transformers for language understanding.
Bert: 言語理解のための双方向トランスフォーマーの事前トレーニング。
0.80
arXiv preprint arXiv:1810.04805, 2018.
arXiv preprint arXiv:1810.04805, 2018
0.39
[36] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, In Proceedings of the 31st
[36]Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, in Proceedings of the 31th 訳抜け防止モード: 36 ]アッシュ・ヴァシワニ ノアム・シャザー ニキ・パルマー jakob uszkoreit , llion jones , aidan n gomez , in proceedings of the 31st
0.58
Łukasz Kaiser, and Illia Polosukhin.
シュカシュ・カイザーとイリア・ポロスキン。
0.31
Attention is all you need.
注意はあなたが必要とするすべてです。
0.63
International Conference on Neural Information Processing Systems, pages 6000–6010, 2017.
Qnlp in practice: Running compositional models of meaning on a quantum computer.
Qnlp in practice: 量子コンピュータ上で意味の合成モデルを実行する。
0.80
arXiv preprint arXiv:2102.12846, 2021.
arXiv preprint arXiv:2102.12846, 2021
0.40
[41] Michael A. Nielsen and Isaac Chuang.
マイケル・A・ニールセンとアイザック・チュアン。
0.42
Quantum Computation and Quantum Information.
量子計算と量子情報。
0.71
Amer- ican Journal of Physics, 70(5):558–559, May 2002.
アマー ican Journal of Physics, 70(5):558–559, 5月。
0.47
[42] Charles A Micchelli, Yuesheng Xu, and Haizhang Zhang.
42] チャールズ・ア・ミチェリ、ユエシェン・クウ、ハイジャン・ジャン
0.40
Universal kernels.
ユニバーサルカーネル。
0.55
Journal of Machine
Journal of Machine(英語)
0.55
Learning Research, 7(12), 2006.
平成18年(2006年)7月、卒業。
0.41
[43] Riccardo Di Sipio, Jia-Hong Huang, Samuel Yen-Chi Chen, Stefano Mangini, and Marcel Worring.
[43]Riccardo Di Sipio、Jia-Hong Huang、Samuel Yen-Chi Chen、Stefano Mangini、Marcel Worring。
0.39
The dawn of quantum natural language processing.
量子自然言語処理の夜明け。
0.60
In ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 8612–8616.
icassp 2022-2022 ieee international conference on acoustics, speech and signal processing (icassp) には8612-8616ページがある。
0.68
IEEE, 2022.
IEEE、2022年。
0.76
[44] Eric R. Ziegel, E. L. Lehmann, and George Casella.
[44]エリック・r・ジーゲル、e・l・リーマン、ジョージ・カゼラ
0.63
Theory of Point Estimation.
ポイント推定の理論。
0.71
Technometrics,
テクノメトリックス。
0.54
41(3):274, Aug 1999.
41(3):274、1999年8月。
0.74
[45] L´eon Bottou.
45]l eon bottou。
0.47
Stochastic Learning. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), volume 3176, pages 146–168.
Frontiers of Data and Domputing, 1(1):105–115, 2019.
データとドミネーションのフロンティア、2019年1(1):105–115。
0.66
[50] Dheeru Dua and Casey Graff.
[50]Dheeru DuaとCasey Graff。
0.38
UCI machine learning repository, 2017.
UCI機械学習レポジトリ、2017年。
0.79
[51] Diederik P. Kingma and Jimmy Lei Ba.
51]ディーデリク・p・キングマとジミー・ライ・バ
0.47
Adam: A method for stochastic optimization.
Adam: 確率最適化の方法です。
0.69
3rd International Conference on Learning Representations, ICLR 2015 - Conference Track Proceedings, Dec 2015.
3rd international conference on learning representations, iclr 2015 - conference track proceedings, dec 2015 (英語)
0.45
[52] Ian Goodfellow, Yoshua Bengio, and Aaron Courville.
52] イアン・グッドフェロー、ヨシュア・ベンジオ、アーロン・クールヴィル
0.59
Deep Learning.
ディープラーニング。
0.39
MIT Press, 2016.
2016年、MIT出版。
0.65
http: //www.deeplearningbo ok.org.
http: deeplearningbook.org (英語)
0.37
16
16
0.42
英語（論文から抽出）
日本語訳
スコア
Appendix for Quantum Self-Attention Neural Networks
量子自己アテンションニューラルネットワークの付録
0.79
for Text Classification
テキスト分類のために
0.72
1 Analytical Gradients Here, we give the stochastic analytical partial gradients of the loss function with regard to its parameters as follows.
1 分析勾配 ここでは,損失関数の確率的解析的部分勾配を,そのパラメータについて下記のように示す。
0.58
We first consider the parameters in the last quantum self-attention neural network layer, i.e., θ(L) , and the final fully-connected layer, i.e., w, b, and then the parameters in the front layers could be evaluated in a similar way and be updated through back-propagation algorithm [52].
i , (S8) where the four terms denote the residual, value, query and key parts, respectively, and each sub-term can be evaluated similarly to the above analysis.