Easy-to-understand LLM (Part 2)

Preface
1. Large model activation function
- 1、ReLU
- 2、GeLU
- 3、GLU
- 4、GeGLU
- 5、SwiGLU
2. Position coding
- 1. Rotation position encoding
3. Decoder-only model
Summarize

Preface

Book continues above.

1. Large model activation function

Refer to the large model for structural details .

1、ReLU

The original Transformer uses ReLU, and the activation function used by T5 and OPT is also ReLU. At this time, FFN can be expressed as: FFN (
$FFN(x,W_{1},W_{2},b_{1},b_{2})=ReLU(xW_{1}+b_{1})W_{2 }+b_{2}$

2、GeLU

The activation function used by GPT-1, GPT-2, GPT-3, and BLOOM is GeLU (Gaussian Error Linear Unit). The expression of GeLU is as follows:
$GeLU(x)=x\Phi(x)$
where:
$\Phi(x)=\int_{-\infty}^{x}\frac{e^{-t^ {2}/2}}{\sqrt{2\pi}}dt$
Two approximate calculations of GeLU are given in the original paper:
$x\Phi(x)\approx x\sigma(1.702x)$
whereσ $\sigma$ 是 $s i g m o i d$ function. Or:
$x\Phi(x)\approx \frac{1}{2}x[1+tanh(\ sqrt\frac{2}{\pi}(x+0.044715x^{3}))]$
At this time, FFN can be expressed as:
$FFN(x,W_{ 1},W_{2},b_{1},b_{2})=GeLU(xW_{1}+b_{1})W_{2}+b_{2}$

3、GLU

GLU Variants Improve Transformer proposes that the activation function can be improved by using gated linear units: GLU (Gated Linear Units). First of all, the basic form of GLU is: do a bilinear transformation on the input (an extra matrix V), and add $s i g m o i d$ 变换：
$GLU(x,W,V,b,c)=\sigma(xW+b)\otimes(xV+c)$
At this time, FFN can be expressed as:
$FFN(x,W_{1},W_{2},V,b_{1},b_{2},c)=(\sigma(xW_{1}+b_{1})\otimes(xV +c))W_{2}+b_{2}$

4、GeGLU

in GLU $If s i g m o i d$ is replaced by the GeLU activation function, a variant of GLU is obtained: GeGLU. The activation function used by GLM-130B is GeGLU. The specific expression is as follows:
$GeGLU(x,W,V,b,c)=GeLU(xW+b)\otimes(xV+c)$
At this time, FFN can be expressed as:
$FFN(x,W_{1},W_{2},V,b_{1},b_{2},c)=(GeLU(xW_{1}+b_{1})\otimes( xV+c))W_{2}+b_{2}$

5、SwiGLU

in GLU $If s i g m o i d$ is replaced with another activation function, another variant of GLU is obtained: SwiGLU. The activation function used by LLaMA is SwiGLU. The specific expression is as follows:
$SwiGLU(x,W,V,b,c)=Swish_{\beta}(xW+b)\otimes(xV +c)$
At this time, FFN can be expressed as:
$FFN(x,W_{1},W_{2},V,b_{1},b_{2},c)=(Swish_{\beta}(xW_{1}+b_{1} )\otimes(xV+c))W_{2}+b_{2}$
其中， $Swish_{\beta}(x)=x\cdot\sigma(\beta x)$ ， $\beta$ is a specified constant, usually 1. Compared with the original ReLU activation function, since it involves a linear representation of one more dimension, it will face an increase in the number of parameters and an increase in the amount of calculations. And how does LLaMA overcome this? $W_{1}, W_{2}, V$ in SwiGLU $W_{1} 、 W_{2},$ the matrix dimension of $V$ $(d im, d im)$ becomes $\frac{2}{3}dim)$ , thereby equalizing the amount of parameters and the amount of calculation.

2. Position coding

1. Rotation position encoding

Rotary Position Embedding (RoPE) comes from Su Jianlin's Rotary Transformer . Traditional positional encoding is usually summed with token embedding when inputting. When performing subsequent Attention calculations, the formula is as follows:
$\begin{aligned} QK^{T}&=xW_{Q}\sdot W_{K}^{T}x^{T} +xW_{Q}\sdot W_{K}^{T}p^{T}+pW_{Q}\sdot W_{K}^{T}x^{T}+pW_{Q}\sdot W_{K }^{T}p^{T}\\ O&=A\sdot V=A\sdot(xW_{V}+pW_{V}) \end{aligned}$
其中 $A=softmax(\frac{QK^{T}}{\sqrt{d_{k}}})$ is the attention matrix, $x$ is the input token embedding, $p$ is position embedding.
In fact, the position vector does not necessarily have to be added to the input token embedding to work. From the above formula, we can see that as long as $Q$ 、Adding position coding to the $K$ $A$ , then acts on $V$ on. Rotational position encoding adopts this idea.

Rotation position encoding formula : for $Q$ 's $m$ vectors $q$ 、 $K$ 's $n$ vectors $k$ , inject positional encoding via:
- Theorem : Define the functional function 2.
  $\begin{pmatrix} cos(m\theta) & -sin(m\theta) \\sin(m\theta) & cos(m\theta) \end{pmatrix} \begin{pmatrix} q_{ 1} \\q_{2}\end{pmatrix}$
  $\begin{pmatrix} cos (n\theta) & -sin(n\theta) \\sin(n\theta) & cos(n\theta) \end{pmatrix}\begin{pmatrix}k_{1}\\k_{2}\end {pmatrix}$
  where, $q=\begin{pmatrix} q_{1} \\ q_{2} \end{pmatrix}$ is $Q$ 's $m$ vectors, $k=\begin{pmatrix} k_{1} \\ k_{2} \end{pmatrix}$ is $K$ 's $n$ vectors, $f (q, m)$ 、 $f (k, n)$ after rotation position encoding. $q$ and $k$ . It can be found that the rotation position encoding is actually $q$ and $k$ is multiplied by a rotation matrix respectively (after the vector is multiplied by the rotation matrix, the module (vector size) remains unchanged and the direction changes). This is why it is called rotational position encoding. At this time $QK^{T}$ is (only with $q$ andOne of k: $($
  $\begin {pmatrix} q_{1} & q_{2} then{pmatrix} \begin{pmatrix} cos((mn)\theta) & -sin((mn)\theta) \\sin((mn)\theta) & cos((mn)\theta)\end{pmatrix}\begin{pmatrix}k_{1}\\k_{2}\end{pmatrix}$
- Multidimensional : The above calculation assumes that the word embedding dimension is 2 dimensions, and for $d\ge2$ In the general case of $2$ $The elements in the q$ vector are grouped in pairs, and the same rotation operation is applied to each group. The specific calculation formula is as follows:
  $\begin{pmatrix} cos(m\theta_{0}) & -sin(m\theta_{0}) & 0 & 0 & \cdots & 0 & 0 \\ sin(m\theta_{0}) & cos(m\theta_{0}) & 0 & 0 & \cdots & 0 & 0 \\ 0 & 0 & cos(m\theta_{1}) & -sin(m\theta_{1}) & \cdots & 0 & 0 \\ 0 & 0 & sin(m\theta_{1}) & cos(m\theta_{1}) & \cdots & 0 & 0 \\ \vdots & \vdots & \vdots & \vdots & \ddots & \vdots & \vdots \\ 0 & 0 & 0 & 0 & \cdots & cos(m\theta_{d/2-1}) & -sin(m\theta_{d/2-1}) \\ 0 & 0 & 0 & 0 & \cdots & sin(m\theta_{d/2-1}) & cos(m\theta_{d/2-1}) \\ \end{pmatrix} \begin{pmatrix} q_{0} \\ q_{1}\\ q_{2}\\ q_{3}\\ \vdots\\ q_{d-2}\\ q_{d-1} \end{pmatrix}$
  Among them, $\theta_{j}=10000^{-2j/d}$ ， $j\in \{0,1,\dots,d/2-1\}$ 。 $The calculation of k$ is similar and will not be shown here.
- Efficient calculation : Due to the sparsity of the above-mentioned rotation matrix, directly using matrix multiplication to implement it will be a waste of computing power. It is recommended to implement RoPE in the following way:
  $\begin{pmatrix} q_{0} \\ q_{1}\\ q_{2}\\ q_{3}\\ \vdots\\ q_{d-2}\\ q_{d-1} \end{pmatrix}\otimes \begin{pmatrix} cos(m\theta_{0}) \\ cos(m\theta_{0}) \\ cos(m\theta_{1}) \\ cos(m\theta_{1}) \\ \vdots \\ cos(m\theta_{d/2-1}) \\ cos(m\theta_{d/2-1}) \\ \end{pmatrix}+ \begin{pmatrix} -q_{1} \\ q_{0}\\ -q_{3}\\ q_{2}\\ \vdots\\ -q_{d-1}\\ q_{d-2} \end{pmatrix}\otimes \begin{pmatrix} sin(m\theta_{0}) \\ sin(m\theta_{0}) \\ sin(m\theta_{1}) \\ sin(m\theta_{1}) \\ \vdots \\ sin(m\theta_{d/2-1}) \\ sin(m\theta_{d/2-1}) \\ \end{pmatrix}$
- Intuitive display : There is a very intuitive picture in the paper showing the process of rotation transformation.
- Remote attenuation : RoPE is somewhat similar in form to the triangular position encoding of traditional transformers, except that the triangular position encoding is additive, while RoPE can be regarded as multiplicative. At $\theta_{i}$ In the selection $i_{j} = 1000 0^{- 2 j / d}$ ， $j\in\{0,1,\dots,d/2-1\}$ , which can bring certain remote attenuation. As shown in the figure below:
  
  From the figure we can see that asthe relative distance increases, the inner product result has a tendencyto attenuate. Therefore, choose $\theta_{j}=10000^{-2j/d}$ , which can indeed bring a certain degree of long-range attenuation. In the paper, we also tried to use $\theta_{j}=10000^{-2j/d}$ is initialization, and $\theta_{j}$ Treat it as a trainable parameter, and then find $\theta_{j} after training for a period of time$ There is no significant update, so simply fix $\theta_{j}=10000^{-2j/d}$ .
- RoPE practice : Take a look at the source code of LLaMA and ChatGLM later.
  - ChatGLM2: every $q$ 、 $Only the first half of the k$ vector has rotation position coding added;
  - LLaMA: Implemented Su Jianlin's version of rotation position encoding.
- Advantages and Disadvantages :
  - Advantages : RoPE implements relative position encoding through absolute position encoding. The advantage is that this method can not only process position information, but also distance information, because the rotation operation can well reflect the relative position relationship between elements;
  - Disadvantages : The disadvantage is that it requires more computational resources because the rotation operation is more complex than simple vector addition.
- Reference materials : Understand Rotary Encoding (RoPE) in ten minutes , and understand the rotational position encoding in LLaMA in one article .

3. Decoder-only model

1. Generate tasks

Conditional text generation : When performing a text generation task, pre-generation conditions are given. For example, the input is: given text content: Spring is so beautiful, continue writing the story;
Unconditional text generation : When performing a text generation task, no prerequisite generation conditions are given. For example, the input is: write a composition.

2. Reasoning process

Casual LM：
- Original input composition : For GPT-like models, whether it is a conditional text generation task or an unconditional text generation task, the original input is fed into the model as a whole;
- New input composition : generate tokens one by one, and the generated tokens will be spliced behind the original input to form a new input feeding model, and continue to generate new tokens;
- Attention calculation : The Mask Attention mechanism is satisfied between inputs (the later token can see the previous token, and the previous token cannot see the later token);
- Token generation : The new token generation mechanism is to use the last column of logits in the output layer for full vocabulary classification; there are many ways to select which word to output, please refer to the decoding and generation method below .
Prefix LM：
- Original input composition : For GLM-like models, whether it is a conditional text generation task or an unconditional text generation task, the original input is fed into the model as a whole;
- New input composition : generate tokens one by one, and the generated tokens will be spliced behind the original input to form a new input feeding model, and continue to generate new tokens;
- Attention calculation : The inputs satisfy the Prefix Attention mechanism (the original input tokens can see each other, and the newly spliced token can only see the previous token, but not the following token);
- Token generation : The new token generation mechanism is to use the last column of logits in the output layer for full vocabulary classification. There are many ways to select which word to output. Please refer to the decoding and generation method below .

3. Decoding generation method

Greedy Search : Greedy search.
- Method : Select the word with the highest probability at each time step;
- Parameter settings : do_sample = False, num_beams = 1;
- Disadvantages :
  - Generate text repetition;
  - Generating multiple results is not supported. When num_return_sequencesthe parameter setting is greater than 1, the code will report an error saying that greedy search does not support this parameter greater than 1.
Beam Search : Beam search, an improvement on the greedy strategy, will degenerate into a greedy search strategy when num_beams = 1.
- Method : Select the num_beams words with the highest probability at each time step;
- Parameter settings : do_sample = False, num_beams > 1;
- Disadvantages : Although the results are smoother than greedy search, there is still the problem of generating duplicates.
Top-K sampling：
- Method : At each time step, top-k tokens will be retained, then the probabilities of the top-k tokens will be renormalized, and finally sampling will be performed among the renormalized k tokens;
- Parameters : do_sample = True, num_beams = 1, setting top_k;
- Disadvantages : When the distribution is steep, tokens with low probability will still be sampled, or when the distribution is flat, only some of the available tokens can be sampled;
Top-P sampling：
- Method : At each time step, the tokens are sorted from high to low according to their probability of occurrence. When the sum of probabilities is greater than top-p, subsequent samples will not be taken. Then re-normalize the probabilities of these tokens and perform sampling;
- Parameters : do_sample = True, num_beams = 1, setting top-p, value between 0-1;
- Note : The top-p sampling method is often used in combination with the top-k sampling method. Each time the smallest sampling range of the two is selected for sampling, it can reduce the chance of sampling a very small probability token when the prediction distribution is too flat.
Temperature sampling：
- Method : Strictly speaking, temperature adjustment is not a sampling method, but a probability changing method that needs to be combined with other sampling methods to work. Specifically, the temperature is adjusted by softmax in to $so$ $f$ $t$ $ma$ $x$ $t$ to change the probability of each token output:
  $softmax(x_{i})=\frac{e^{x_{i}}/t}{\sum_ {j=1}^{n}e^{x_{j}}/t}$
  Among them, $n$ is the vocabulary size, $x_{j}$ Indicates the logits value corresponding to each token in the vocabulary, $x_{i}$ is the logits value of the current token, $t$ is the temperature coefficient.
- Parameters : temperature;
- Note : It can be seen from the above formula that when $When t$ approaches 0, the difference in probabilities between tokens will increase, and the probability distribution of all tokens will be steeper, as shown in the figure below. If the top-p sampling mode is adopted, the optional range of final output tokens will be narrowed (because only fewer tokens and probability sums are needed to meet the conditions), and the model output will be more stable; when $When t$ approaches infinity, the probability difference between high-probability tokens will be reduced, and the probability distribution of all tokens will be flatter. That is to say, the optional range of final output tokens will increase, and the model output will be more random.

Summarize

To understand LLM, just read this article! ! !