[Natural Language Processing | Transformer] Transformer: Attention is All You Need paper explanation

Transformer is proposed by the paper "Attention is All You Need":

The paper address is:

https://arxiv.org/pdf/1706.03762.pdf

1. The overall structure of Transformer

First introduce the overall structure of Transformer . The following is the overall structure of Transformer for Chinese-English translation:

The picture above is the overall structure of the Transformer , the Encoder on the left and the Decoder on the right .

It can be seen that the Transformer consists of two parts, the Encoder and the Decoder . Both the Encoder and the Decoder contain 6 blocks . The workflow of Transformer is roughly as follows:

Step 1 : Obtain the representation vector X of each word in the input sentence. X is obtained by adding the Embedding of the word ( Embedding is the Feature extracted from the original data ) and the Embedding of the word position .

The second step : the obtained word representation vector matrix (as shown in the figure above, each row is a representation of a word $x$ ) is passed intothe Encoder, and after 6Encoder blocksCCof all words in the sentence can be obtained$C$ , as shown in the figure below. The word vector matrix uses$X_{n\times d}$said, $n$ is the number of words in the sentence,$d$ is the dimension representing the vector ($d=512$ ). The matrix dimension output byeachEncoder block

The figure above shows the Transformer Encoder encoding sentence information.

The third step : the encoding information matrix CC output by the Encoder$C$ is passed tothe Decoder,and the Decoderi + 1 i + 1according to the currently translated word 1~ i$i+1$ , as shown in the figure below. In use, translate to word$i+When\; it\; is\; 1$ , it is necessary to coveri + 1 i + 1throughthe Mask$i+$Words after$1\; .$

Transofrmer Decoder predictions.

The Decoder in the above picture received the encoding matrix CC of the Encoder$C$ , then first input a translation start character "", predict the first word "I"; then input the translation start character "" and the word "I", predict the word "have", and so on. This isTransformer, followed by the details of each part.

2. Input of Transformer

The input of word in Transformer means $x$ is obtainedby adding the wordEmbeddingand the positionEmbedding(Positional Encoding

2.1 Word Embedding

There are many ways to obtain the Embedding of a word . For example, it can be obtained by pre-training algorithms such as Word2Vec and Glove , or it can be obtained by training in Transformer .

2.2 Position Embedding

In Transformer , in addition to word Embedding , positional Embedding also needs to be used to indicate the position where the word appears in the sentence. Because Transformer does not use the structure of RNN , but uses global information, it cannot use the order information of words, and this part of information is very important for NLP .

So Transformer uses position Embedding to save the relative or absolute position of words in the sequence.

The position Embedding is represented by PE, and the dimension of PE is the same as the word Embedding . PE can be obtained through training, or it can be calculated using some formula. The latter is adopted in Transformer, and the calculation formula is as follows: PE (
$$\begin{gathered} P{E}_{(pos,2i}=\sin (pos/10000^{2i/d}) \\ P{E}_{(pos,2i+1}=\cos (pos/10000^{2i/d}) \end{gathered}$$
Among them, pos represents the position of the word in the sentence,$d$ represents the dimension of PE (same as the word Embedding),$2i$ means even dimension,$2i\_+1$ means odd dimension (ie$2i\_\le d$，$2i\_+1\le d$ ). Using this formula to calculate PE has the following benefits:

• Make PE able to adapt to sentences longer than all the sentences in the training set. Suppose the longest sentence in the training set has 20 words, and suddenly a sentence with a length of 21 comes, then the 21st sentence can be calculated by using the formula calculation method bit Embedding .
• allows the model to easily calculate the relative position, for a fixed-length spacing $k$，$P{E}_{(pos+k)}$You can use $P{E}_{(pos)}$calculated.

因为 $\mathrm{without}(A)+B)=\mathrm{without}\left(A\right)\cos (B)+\cos (A)\sin (B),\cos (A+B)=\cos (A)\cos (B)-\mathrm{without}\left(A\right)\sin (B)$。

Add the word Embedding and the position Embedding of the word to get the representation vector $x$，$x$ isTransformer.

3. Self-Attention (self-attention mechanism)

The picture above is the internal structure diagram of the Transformer in the paper , with the Encoder block on the left and the Decoder block on the right .

The part in the red circle is Multi-Head Attention , which is composed of multiple Self-Attentions . You can see that the Encoder block contains a Multi-Head Attention , and the Decoder block contains two Multi-Head Attentions (one of which uses Masked ). An Add & Norm layer is also included above the Multi-Head Attention . Add means Residual Connection ( Residual Connection ) is used to prevent network degradation, and Norm means Layer Normalization , which is used to normalize the activation value of each layer.

Because Self-Attention is the focus of Transformer , we focus on Multi-Head Attention and Self-Attention . First, we will learn more about the internal logic of Self-Attention .

3.1 Self-Attention structure

The above picture shows the structure of Self-Attention , and the matrix QQ is needed in the calculation$Q$ (query),$K$ (key value),$V$ (value).

In practice, Self-Attention receives the input (word representation vector xxThe matrix XXcomposed of$x$$X$ ) or the output of the previousEncoder block. while$Q$, $K$, $V$ isobtained by linear transformation through the input ofSelf-Attention

3.2 Calculation of Q, K, V

The input of Self-Attention uses matrix $X$ is represented, you can use the linear transformation matrix$W_Q$, $W_K$, $W_V$Calculate $Q$, $K$, $V.$ _ The calculation is shown in the figure below, pay attention to$X$, $Q$, $K$, V V Each row of$V\; represents\; a\; word.$

3.3 Output of Self-Attention

get the matrix $Q$, $K$, $After\; V$ , the output ofSelf-Attentioncan be calculated
$$Attention(Q,K,V)=softmax(\frac{Q{K}^T}{\sqrt{d_k}})$$
$d_k$is $Q$，$The\; number\; of\; columns\; of\; the\; K$ matrix, that is, the vector dimension.

Calculate the matrix $Q$ and$The\; inner\; product\; of\; each\; row\; vector\; of\; K$ , in order to prevent the inner product from being too large, it is divided by$d_k$square root of . $Q$ timesKKAfter the transposition of$K$$n$，$n$ is the number of words in the sentence, and this matrix can represent the attentionbetween words. The picture below is$Q$ times$K^T$ , 1234 represents the words in the sentence.

【$Q$ timesKKCalculation of transpose of$K\; ]$

Get $Q{K}^{After\; T}$ , useSoftmaxthe attentionof each word for other wordsThe Softmaxin the formulais to performSoftmax, that is, the sum of each row becomes 1.

[Perform Softmax on each row of the matrix ]

After getting the Softmax matrix, it can be combined with $V$ is multiplied to get the final output$Z$。

【Self-Attention output】

The first row of the Softmax matrix in the above figure represents the attention coefficient of word 1 and all other words , and the final output of word 1 $Z_1$equals all words $i$ 's value$V_i$According to the ratio of attention coefficients added together, as shown in the figure below:

[ $Z_i$calculation method]

3.4 Multi-Head Attention

In the previous step, we already know how to calculate the output matrix ZZ through Self-Attention$Z$ , whileMulti-Head Attentionformedby a combination of multipleSelf-AttentionsMulti-Head Attention:

From the above figure, you can see that Multi-Head Attention contains multiple Self-Attention layers, first input $X$ is passed to hhrespectively$Among\; h$ differentSelf-Attentionshhis calculated$h$ output matrices$Z.$ _ The figure below is$h=In\; the\; case\; of\; 8$ , 8 output matrices$Z$。

Get 8 output matrices $Z_1$to $Z_8$After that, Multi-Head Attention stitches them together ( Concat ), and then passes it into a Linear layer to get the final output ZZ of Multi-Head Attention$Z$。

You can see the matrix ZZ output by Multi-Head Attention$Z$ and its input matrix$The\; dimensions\; of\; X$ are the same.

4. Encoder structure

【Transformer Encoder block】

The red part in the above figure is the Encoder block structure of the Transformer , which can be seen to be composed of Multi-Head Attention , Add & Norm , Feed Forward , Add & Norm . I have just learned about the calculation process of Multi-Head Attention , now let's take a look at the Add & Norm and Feed Forward parts.

4.1 Add & Norm

The Add & Norm layer consists of Add and Norm . The calculation formula is as follows:
$$LayerNorm(X+MultiHeadAttention(X))$$
$$LayerNorm(X+FeedForward(X))$$
where$X$ representsMulti-Head AttentionorFeed Forward,MultiHeadAttention($X$ ) andFeedForward($X$ ) means output (output and input$The\; X$ dimension is the same, so it can be added).

Add refers to $X$ + MultiHeadAttention($X$ ), is a residual connection, usually used to solve the problem of multi-layer network training, allowing the network to only focus on the current difference,often used inResNet

Norm refers to Layer Normalization , which is usually used in the RNN structure. Layer Normalization will convert the input of each layer of neurons into the same mean and variance, which can speed up convergence.

4.2 Feed Forward

The Feed Forward layer is relatively simple. It is a two-layer fully connected layer. The activation function of the first layer is Relu , and the activation function is not used in the second layer. The corresponding formula is as follows:
$$\max (0,X{W}_1+b_1)W_2+b_2$$
$X$ is the input, andthe dimension of the output matrix obtained byFeed Forward$X$ agrees.

4.3 Composition of Encoder

An Encoder block can be constructed through the Multi-Head Attention , Feed Forward , Add & Norm described above , and the Encoder block receives the input matrix X ( n × d ) X_{(n \times d)}$X_{(n\times d)}$, and output a matrix $O_{(n\times d)}$. An Encoder can be formed by superimposing multiple Encoder blocks .

The input of the first Encoder block is the representation vector matrix of sentence words, the input of the subsequent Encoder block is the output of the previous Encoder block , and the matrix output by the last Encoder block is the coding information matrix $C$ , this matrix will be used in Decoderlater.

【Encoder encoding sentence information】

Five, Decoder structure

The red part in the above figure is the Decoder block structure of the Transformer , which is similar to the Encoder block , but there are some differences:

• Contains two Multi-Head Attention layers;
• The first Multi-Head Attention layer uses the Masked operation;
• KK of the second Multi-Head Attention layer$K$, $V$ matrix usesEncoder's encoding information matrix$C$ performs calculations, while$Q$ is calculated using the output of the previousDecoder block;
• Finally there is a Softmax layer that calculates the probability of the next translated word.

5.1 The first Multi-Head Attention

The first Multi-Head Attention of the Decoder block uses the Masked operation, because it is translated sequentially during the translation process, that is, after the translation of the secondOnly$i$$i+1$ word. The second iibythe Maskedoperation$i$ words know$i+$Information after$1\; word.$Let's take the translation of "I have a cat" into "I have a cat" as an example to understand theMaskedoperation.

The following description uses a concept similar to Teacher Forcing. Children's shoes who are not familiar with Teacher Forcing can refer to the following article for a detailed explanation of the Seq2Seq model. In the Decoder , it is necessary to solve the current most likely translation based on the previous translation, as shown in the figure below. First, the first word is predicted as "I" based on the input "", and then the next word "have" is predicted based on the input "I".

【Decoder prediction】

Decoder can use Teacher Forcing and parallelize training during training, that is, pass the correct word sequence (I have a cat) and corresponding output (I have a cat) to Decoder . Then in predicting the $When\; the\; i$ is output, the$i+The\; words\; after\; 1$ are covered up. Note thatthe Maskoperation isused beforethe SoftmaxofSelf-Attention

The first step : Decoder 's input matrix and Mask matrix, the input matrix contains the representation vector of five words "I have a cat" (0, 1, 2, 3, 4), and the Mask is a $5\times$Matrix of$5\; .$InMask,it can be found that word 0 can only use the information of word 0, while word 1 can use the information of words 0 and 1, that is, only the previous information can be used.

[Input matrix and Mask matrix]

Step 2 : The next operation is the same as the previous Self-Attention , by inputting the matrix $X$ is calculated to get$Q$, $K$, $V$ matrix. Then calculate$Q$ and$K^{}$The product of${}^T$$Q{K}^T$。

【 $Q$ timesKKTranspose of$K\; ]$

The third step : get $Q{K}^{}$After${}^T$ , Softmaxto calculatethe attention scoreSoftmax, weneed to usethe Maskmatrix to block the information after each word. The blocking operation is as follows:

[MaskbeforeSoftmax]

Get Mask $Q{K}^{After\; T}$ inMask $Q{K}^{}$Softmaxis performed on${}^T$ , and the sum of each row is 1. But word 0 has an attention scoreon words 1, 2, 3, and 4.

Step 4 : Use Mask $Q{K}^T$ and matrix$V$ is multiplied to get the output$Z$ , then the output vector$Z_1$It contains only word 1 information.

【Output after Mask 】

Step 5 : Through the above steps, an output matrix Z i Z_i of Mask Self-Attention can be obtained$Z_i$, and then similar to the Encoder , multiple output Z i Z_i are spliced ​​​​by Multi-Head Attention$Z_i$Then calculate the output ZZ of the first Multi-Head Attention$Z$，$Z$ with input$Same\; as\; the\; X$ dimension.

5.2 The second Multi-Head Attention

The second Multi-Head Attention of the Decoder block has not changed much, the main difference is the KK of the Self-Attention$K$, $The\; V$ matrix is ​​not calculatedusing the output of the previousDecoder blockthe encoding information matrixCCthe Encoder$C$ calculated.

According to the output CC of Encoder$C$ calculates$K$, $V$ , according to the output ZZofthe previousDecoder block$Z$ calculates$Q$ (if it is the firstDecoder block, use the input matrix$X$ for calculation), the subsequent calculation method is consistent with the previous description.

The advantage of this is that during the Decoder , each word can use the information of all the words of the Encoder (the information does not need a Mask ).

5.3 Softmax predicts output words

The last part of the Decoder block is to use Softmax to predict the next word. In the previous network layer, we can get a final output $Z$ , because ofMask, the output of word 0 is$Z_0$Contains only word 0 information, as follows:

[ ZZ before Decoder Softmax$Z$】

Softmax predicts the next word based on each row of the output matrix:

【Decoder Softmax prediction】

This is the definition of the Decoder block . Like the Encoder , the Decoder is composed of multiple Decoder blocks .

6. Summary of Transformer

• Transformer is different from RNN and can be trained in parallel better;
• Transformer itself cannot use the order information of words, so you need to add position Embedding to the input , otherwise Transformer is a bag of words model;
• The focus of Transformer is the Self-Attention structure, which uses $Q$, $K$, $The\; V$ matrix is ​​obtained by linear transformation of the output;
• There are multiple Self-Attentions in Multi-Head Attention in Transformer , which can capture the attention score of the correlation coefficient in multiple dimensions between words .