Computer LCG/PCG/MWC/XorShift and other PRNG algorithms, as well as the implementation of random algorithms such as Math.random() in V8 and crypto in webkit

This article is long. If you want to see the random number implementation of js directly, you can locate the ECMAScript JavaScript implementation chapter of this article .

*My github: https://github.com/MichealWayne , my blog address: https://blog.michealwayne.cn/

introduction

No matter which programming language is used for development, there are many scenarios that require the generation of random numbers. Of course, in the daily development process, we will directly call API methods, such as js and Math.random()Python random(), but the principle of implementing random numbers is also a problem worthy of further study.

Application scenarios of random numbers

Simulation : When using a computer to simulate natural phenomena, you need to use random numbers to make things more realistic, such as simulating the flow of people at an airport.
Sampling : It is often impractical to examine all possible situations, and random sampling will allow us to understand "typical" behavior.
Numerical Analysis : Many clever techniques have been devised using random numbers to solve complex numerical problems.
Computer Programming : To verify the effectiveness of computer algorithms, random values are a good source of data. More importantly, random values are critical to the operation of randomized algorithms, which are often much superior to deterministic algorithms.
Decision-making : Many decision-makers make judgments by tossing coins or throwing darts. It is important to make such completely "unbiased" judgments. Randomness is also an indispensable part of the optimization strategy in matrix game theory.
Aesthetics : A bit of randomness can make computer-generated graphics and music seem more alive.
Entertainment : Rolling dice, shuffling playing cards, spinning roulette and other entertainment methods. These traditional uses of random numbers have been named "Monte Carlo methods", which has become a general term to describe any algorithm that uses random numbers.

Precisely because it has so many applications, the implementation of random numbers is particularly important.

Random number concept

probability theory

First, let’s review the concepts related to random numbers in probability theory in college:

Frequency : Defined under the same conditions, nan experiment was conducted. In this nexperiment, the number of times an event Aoccurred nAis called Athe frequency of the event, and the ratio is called the frequency of nA/nthe event .A
Probability : Let Ebe a random experiment and Sbe its sample space. For Eeach event of , Aassign a real number, recorded as P(A), called the probability of the event.
Random variable : Assume that the sample space of the random experiment is S={e}, if is a real-valued single-valued function X=X(e)defined on the sample space , it is called a random variable.SX=X(e)
Random events : SEvents that may or may not occur under conditions are called Srandom events relative to conditions.

An independent random number sequence with a certain distribution: the occurrence of each number is only by chance and has nothing to do with other numbers in the sequence. Each number has a certain probability of entering any given range.

Whether a random event occurs or not in an experiment is random, but randomness contains regularity. The probability of random event A is the stable value of frequency, and frequency is an approximation of probability.

Like the one in js, Math.random()it belongs to continuous and discrete uniform distribution in probability distribution theory. The characteristic of a uniform distribution is that all basic events are equally likely.

judgement standard

You can refer to the Federal Office for Information Security (German Federal Office for Information Security) which gives four standards for evaluating the quality of random number generators - BSI evaluation standards:

K1 - The probability that the generated random number sequences are different from each other should be high.
K2 - According to some statistical tests, the generated sequence cannot be distinguished from a truly random sequence. These tests include: monobit test (equal number of 0s and 1s in the sequence), poker test (a special case of the chi-squared test), runs test (frequency of runs of different lengths), longruns test from BSI and NIST, autocorrelation test.
K3 - Given any subsequence, no attacker can compute subsequent sequences or the internal state of the generator.
K4 - Given the internal state of a generator, no attacker can compute the previous sequence or the previous state of the generator.
For cryptographic applications, only generators that meet the K3 and K4 standards are acceptable. .

computer implementation

Deterministic methods cannot truly create randomness, as stated in "The Art of Computer Programming (Volume 2)" : "Most random number generators in use today are not good enough, and developers tend to use them as they come. , without understanding the specific generation strategy. So we often find that some slightly flawed and old random number generators are blindly used in one program after another, and no one cares about their limitations. .”

development path

In the early days, those who needed random numbers in scientific work used methods such as grabbing balls from a "fully rotated" jar or using dice or cards to obtain random numbers.
In 1927, LHC Tippett published a table of more than 40,000 random numbers "taken at random from census reports." Since then, special devices have been built to mechanically generate random numbers.
In 1939, MG Kendall and B. Babington-Smith used the first such machine to create a table of 100,000 random numbers.
In 1949, DHLehmer proposed the algorithm scheme of linear congruence method (LCG).
The Ferranti Mark I computer, first installed in 1951, had a built-in instruction that used a resistor noise generator to put 20 random bits into the accumulator, a feature recommended by AMTuring.
In 1955, the RAND Company published a widely used table of 1 million random numbers, which was obtained using another special device. A famous random number machine named ERNIE was used for many years to generate the winning numbers for British government bonds.
Shortly after the advent of computers, people began to explore effective methods for finding random numbers in computer programs. A table could be used, but due to memory space and input time requirements, the practicality of this method was limited. Technological developments in the 1990s made this table useful again, as 1GB of tested random numbers could be distributed (stored) on CDROM.
In 1995, George Marsaglia promoted random numbers by superimposing the current output of a noise diode with certain scrambled rap music to generate 650MB of random values (called black and white noise) and making them into a demo CD. The renaissance of the watch.
In 2014, the computer PRNG algorithm-PCG was born, which achieved excellent statistical performance with smaller, faster code and small state quantities.

Algorithm (pseudo-random)

Centering of Squares Method - Random Sequence

The shortcomings of early mechanical means of generating random numbers led to interest in using common arithmetic operations on a computer to generate random numbers. Around 1946, Jogn von Neumann was the first to suggest using this method. His approach was to square the previous random number and extract the middle number.

For example, if you are generating a 10-digit number and the previous value was 5772156649, squaring it gives you 33317792380594909201, so the next number is 7923805949.

There are very obvious objections to this technique: since each number is completely determined by the number before it, how can the sequence generated in this way be random? This sequence is of course not random, it just seems random. In typical applications, the actual relationship between a number and the number that follows it has no objective meaning, so this non-random characteristic is not really undesirable. Intuitively, squaring seems to mess up the previous numbers quite a bit.

It has been proven that von Neumann's method is not a good way to find random numbers. The danger is that this sequence is prone to short cycles of repeated elements. For example, if 0 once appears as a number in this sequence, it will keep recurring itself.

Some people conducted experiments in the early 1950s with the "square-center method." GE Forsythe used 4 digits instead of 10 digits and experimented with 16 different initial values. It was found that 12 of them resulted in a sequence ending with the cycle 6100, 2100, 4100, 8100, 6100,..., and two of them degenerates into 0. N. Metropolis conducted extensive experiments with the method of squaring, mostly using the binary number system. He demonstrated that when working with 20 digits, the sequence could degenerate into 13 different cycles, the longest of which Period bit 142.

π (Pi)—random sequence

In 1965, according to Dr. IJ Matrix, mathematicians treated the decimal expansion of π as a random sequence that, to a modern numerical logician, was rich in patterns worth noting.

For example, Dr. Matrix pointed out that the first repeated two-digit number in the expansion of π is 26, and its second appearance is in the middle of a wonderful repeating pattern.

p-5

After listing many digit numbers or their other properties, one discovers that π, if interpreted correctly, can reflect the entire history of human experience.

Algorithm K - Super Random Number Generator

p-2 p-3

(where X is a 10-digit decimal number)

Given the complex design of Algorithm K, it seems that this algorithm can produce an infinite number of incredibly random numbers, but in fact, when this algorithm is first put on a computer, it converges almost immediately to the 10-digit value 6065038420 ——Very coincidental, this algorithm converts this number into itself. If it starts with another number, the sequence starts to cycle with a period of length 3178 after 7401 values.

p-4

This thing tells us that random numbers cannot be generated by random selection, and certain theories should be used.

PRNG

The following algorithms are implemented through theoretical algorithms, collectively called PRNG (pseudo random number generator), also known as deterministic random bit generator (DRBG). Among them, a PRNG suitable for cryptographic applications is called a cryptographically secure PRNG (CSPRNG). A necessary condition of CSPRNG is that an enemy that does not know the initial seed has only a negligible advantage in distinguishing the generator's output sequence from a truly random sequence. In other words, if the PRNG only needs to pass certain statistical tests, then the CSPRNG must pass all statistical tests within the polynomial complexity of the seed size.

PRNG is an algorithm that generates a sequence of numbers whose properties approximate those of a sequence of random numbers. The sequence generated by PRNG is not truly random, so it is completely determined by an initial value. This initial value is called the PRNG's random seed (seed, but this seed may contain true random numbers). Although sequences that are close to true randomness can be generated by hardware random number generators, pseudorandom number generators are also important in practice because of their advantages in speed and reproducibility.

The period of a PRNG is defined as: the maximum length of the non-repeating prefix sequence of all initial values. The period is limited by the number of states, usually represented by the number of bits. However, the period length may double with each additional bit, so building a PRNG with a period long enough is easy for many practical applications.

Linear congruential generators ( LCG )

A popular random number generation program in use today is a special case of the following scheme introduced by DHLehmer in 1949. We choose 4 magic integers:

m, 模*数； 0<m
a, 乘数；  0<=a<m
c, 增量；  0<=c<m
X0,开始值; 0<=X0<m

Then by setting formula (1)

Xn+1 = (aX0 + c) mod m,  n>=0

And get the desired random number sequence <Xn>. This sequence is called a linear congruential sequence. Finding the remainder of m is a bit like determining the landing point of the ball on a spinning roulette wheel.

For example, when m = 10 and X0 = a = c = 7, the resulting sequence is:

7,6,9,0,7,6,9,0...

As this example shows, the sequence is not always "random" for all choices of m, a, c and X0. This example has a period of length 4 and illustrates the fact that congruent sequences always enter a cycle.

Improve:

Definition b = a - 1, a>=2, b>=1, formula (2):

Xn+k = (a^k * Xn + (a^k - 1) * c / b) mod m,  k>=0, n>=0

It directly expresses the (n + k)-th term with the help of the n-th term, from which it follows that the <Xn>subsequence consisting of every k-th term is another linear congruential sequence with a multiplier a^k mod m and increment ((a^k - 1) * c / b) mod m.

An important corollary of this formula is that the general sequence defined by m, a, c and X0 can be expressed very simply by means of the special case of c=1 and X0=0, assuming

Y0 = 0, Yn+1 = (a * Yn + 1) mod m

According to formula (2), we get Yk equal to (a^k - 1) / b(modulo m), so the general sequence defined in formula (1) satisfies

Xn = (A * Yn + X0) mod m，其中A = (X0 * b + c) mod m

And it can be concluded through analysis (see "The Art of Computer Programming (Volume 2)" 3.2.1.1 for details) that m can be taken 2^31 - 1( 2^32), the multiplier a is selected between 0.01mand 0.99m, and its binary representation or decimal representation number should not There is a simple normal mode and the recommended value is ( 7^5, that is 16807).

But for modern computers, this cycle is still too short. The internal structure of the generator such as the fence structure and the distribution of hyperplane points are also important. There are specific detection methods for different generators. The most commonly used method for structure detection is spectral testing. Spectral testing is a test based on the maximum distance between adjacent parallel hyperplanes. The larger the distance, the worse the generator.

As shown in the picture:

p-7

MWC - Mersenne rotation algorithm

In 1991, MWC ( multiply-with-carry ) was invented by George Marsaglia and Zaman and is very similar to LCG. In its simplest form, MWC uses a similar formula to a linear congruential generator, but c (the "addend") is different in each iteration.

Its advantage lies in calling a simple computer integer algorithm, so that a random number sequence with a huge cycle can be generated very quickly; the generated cycle period is longer (~), 2^60close 2^2000000to the cycle cycle of the CPU.

It is widely used in game development and is informally known as the "mother of all PRNGs", a name originally coined by Marsaglia himself.

Here is a c language implementation of a small MWC generator that uses 128-bit multiplication and has 64-bit output:

// C99 + __uint128_t MWC, 256 bits of state, period approx. 2^255

/* The state must be neither all zero, nor x = y = z = 2^64 - 1, c =
   MWC_A3 - 1. The condition 0 < c < MWC_A3 - 1 is thus sufficient. */
   
uint64_t x, y, z, c = 1;

#define MWC_A3 0xff377e26f82da74a

uint64_t inline next() {
    
    
    const __uint128_t t = MWC_A3 * (__uint128_t)x + c;
    x = y;
    y = z;
    c = t >> 64;
    return z = t;
}

MWC1616

A high-speed, simple MWC implementation, which was used by the V8 engine of js in the early days (see later).

#define ulong unsigned long

static ulong mwc1616_x = 1;
static ulong mwc1616_y = 2;

void seed_rand_mwc1616(ulong seed) {
    
    
    mwc1616_x = seed | 1;   /* Can't seed with 0 */
    mwc1616_y = seed | 2;
}

ulong rand_mwc1616(void) {
    
    
    mwc1616_x = 18000 * (mwc1616_x & 0xffff) + (mwc1616_x >> 16);
    mwc1616_y = 30903 * (mwc1616_y & 0xffff) + (mwc1616_y >> 16);
    return (mwc1616_x<<16)+(mwc1616_y&0xffff);
}

But MWC failed to pass many tests of TestU01.

PCG

The permuted congruential generator (PCG, permuted congruential generator) is an improvement of LCG, produced in 2014. It achieves excellent statistical performance with smaller, faster code and a small amount of state.
PCG differs from traditional linear congruential generators in the following three aspects:

LCG modulus and states are large, typically twice the required output size;
It uses powers of 2 modulo, which results in a particularly efficient implementation using full-cycle generators and unbiased output bits
The state is not output directly, but the most significant bit of the state is used to select a bitwise rotation or shift, which is applied to the state to produce the output.

It is also the variable rotation that eliminates the problem of the short time period of the low-order bits of the power of 2 LCG.

The PCG range includes many variants. The core LCG has a defined width of 8 to 128 bits, although it is actually recommended to use only 64 and 128 bits. Such as PCG-XSH-RR:

#include <stdint.h>
static uint64_t       state      = 0x4d595df4d0f33173;		// Or something seed-dependent
static uint64_t const multiplier = 6364136223846793005u;
static uint64_t const increment  = 1442695040888963407u;	// Or an arbitrary odd constant

static uint32_t rotr32(uint32_t x, unsigned r)
{
    
    
	return x >> r | x << (-r & 31);
}

uint32_t pcg32(void)
{
    
    
	uint64_t x = state;
	unsigned count = (unsigned)(x >> 59);		// 59 = 64 - 5

	state = x * multiplier + increment;
	x ^= x >> 18;								// 18 = (64 - 27)/2
	return rotr32((uint32_t)(x >> 27), count);	// 27 = 32 - 5
}

void pcg32_init(uint64_t seed)
{
    
    
	state = seed + increment;
	(void)pcg32();
}

XorShift algorithm

The XorShift random number generator, also known as a shift register generator, is a type of pseudo-random number generator discovered by George Marsaglia. It is a subset of Linear Feedback Shift Registers (LFSRs) that allow particularly efficient implementation in software without using overly sparse polynomials.

The basic principle of its implementation is to generate the next number in its sequence by repeatedly taking the XOR of itself or a shifted version of the number, which makes it efficient.

Its shortcomings are also well known and can be corrected by combining them with non-linear functions, such as in xorshift+ or xorshift* generators.

xorshift has three simple modes: 32-bit, 64-bit, 128-bit, as shown in the following c implementation:

#include <stdint.h>

/* 32-bit */
struct xorshift32_state {
    
    
  uint32_t a;
}
uint32_t xorshift32 (struct xorshiift32_state *state) {
    
    
  uint32_t x = state->a;
  x ^= x << 13;
  x ^= x >> 17;
  x ^= x << 5;
  return state->a=x;
}


/* 64-bit */
struct xorshift64_state {
    
    
  uint64_t a;
}
uint64_t xorshift64 (struct xorshift64_state *state) {
    
    
  uint64_t x = state->a;
  x ^= x << 13;
  x ^= x >> 7;
  x ^= x << 17;
  return state->a=x;
}

/* 128-bit */
struct xorshift128_state {
    
    
  uint32_t a, b, c, d;
}
uint32_t xorshift128(struct xorshift128_state *state) {
    
    
  uint32_t t = state->d;
  uint32_t const s = state->a;
  state->d = state->c;
  state->c = state->b;
  state->b = s;
  
  t ^= t << 11;
  t ^= t >> 8;
  return state->a = t ^ s ^ (s >> 19);
}

The 128-bit algorithm passed the stubborn test, but it failed the MatrixRank and LinearComp tests of the BigCrush test suite of the TestU01 framework. All xorshift generators (including the optimizations below) fail some of these tests, but it is easy to perturb the output of such a generator to improve its quality.

xsorrow

Marsaglia suggested that by combining the output with a simple adding counter modulo 2^32(the Weyl sequence), which would double the period 2^32, 2^192-2^32.

#include <stdint.h>
struct xorwow_state {
    
    
  uint32_t a, b, c, d, e;
  uint32_t counter;
}
uint32_t xorwow (struct xorwow_state *state) {
    
    
  uint32_t t = state->e;
  uint32_t s = state->a;
  state->e = state->d;
  state->d = state->c;
  state->c = state->b;
  state->b = s;
  t ^= t >> 2;
  t ^= t << 1;
  t ^= s ^ (s << 4);
  state->a = t;
  state->counter += 362437;
  return t + state->counter
}

A typical example of this generator is the Nvidia CUDA toolkit.

xorshift*

xorshift* takes the xorshift generator and applies reversible multiplication (modulo the word size) to its output as a nonlinear transformation.

The following 64-bit generator with 64-bit state has a maximum period of 2^64-1only TestU01 BigCrush's MatrixRank test fails:

#include <stdint.h>
struct xorshift64s_state {
    
    
  uint64_t a;
}
uint64_t xorshift64s (struct xorshift64s_state *state) {
    
    
  uint64_t x = state->a;
  x ^= x >> 12;
  x ^= x << 25;
  x ^= x >> 27;
  state->a = x;
  return x * UINT64_C(0x2545F4914F6CDD1D);
}

Vigna recommends using the following xorshift1024* generator, which has 1024-bit states and a maximum period of 2^1024-1:

#include <stdint.h>

/* The state must be seeded so that there is at least one non-zero element in array */
struct xorshift1024s_state {
    
    
	uint64_t array[16];
	int index;
};

uint64_t xorshift1024s(struct xorshift1024s_state *state)
{
    
    
	int index = state->index;
	uint64_t const s = state->array[index++];
	uint64_t t = state->array[index &= 15];
	t ^= t << 31;		// a
	t ^= t >> 11;		// b
	t ^= s ^ (s >> 30);	// c
	state->array[index] = t;
	state->index = index;
	return t * (uint64_t)1181783497276652981;
}

xorshift+

Instead of using multiplication, you can also use addition as a faster non-linear transformation. The idea was originally proposed by Saito and Matsumoto (also responsible for Mersenne Twister) in the XSadd generator, which adds two consecutive outputs of the underlying xorshift generator based on 32-bit shifts.

However, the low-order bits of its output have some weaknesses; it failed several BigCrush tests when the output bits were inverted. To solve this problem, Vigna introduces the xorshift+ family based on 64-bit shifts: The following xorshift128+ generator uses 128-bit states with a maximum period of 2^128-1.

#include <stdint.h>

struct xorshift128p_state {
    
    
  uint64_t a, b;
};

/* The state must be seeded so that it is not all zero */
uint64_t xorshift128p(struct xorshift128p_state *state)
{
    
    
	uint64_t t = state->a;
	uint64_t const s = state->b;
	state->a = s;
	t ^= t << 23;		// a
	t ^= t >> 17;		// b
	t ^= s ^ (s >> 26);	// c
	state->b = t;
	return t + s;
}

This generator is one of the fastest to pass BigCrush testing. One drawback of adding continuous output is that the underlying xorshift128 generator is 2D distributed, while the associated xorshift128+ generator is only 1D distributed.

xorshift+ failed BigCrush's reverse testing.

xoshiro sum xoroshiro

xoshiro and xoroshiro are other variations of the shift register generator that use rotation in addition to shifting . According to Vigna, they are faster and produce better quality output than xorshift.
This class of generators has variants with 32-bit and 64-bit integer and floating-point output. For floating point numbers, take the high 53 bits (for binary64) or the high 23 bits (for binary32) because the high bits are of better quality than the low bits in the floating point generator. The algorithm also includes a jump function that sets the state forward by a number of steps (usually a power of 2) to allow many threads of execution to start from different initial states.

xoshiro256** is a general-purpose random 64-bit number generator for the family:

/*  Adapted from the code included on Sebastian Vigna's website */

#include <stdint.h>

uint64_t rol64(uint64_t x, int k)
{
    
    
	return (x << k) | (x >> (64 - k));
}

struct xoshiro256ss_state {
    
    
	uint64_t s[4];
};

uint64_t xoshiro256ss(struct xoshiro256ss_state *state)
{
    
    
	uint64_t *s = state->s;
	uint64_t const result = rol64(s[1] * 5, 7) * 9;
	uint64_t const t = s[1] << 17;

	s[2] ^= s[0];
	s[3] ^= s[1];
	s[1] ^= s[2];
	s[0] ^= s[3];

	s[2] ^= t;
	s[3] = rol64(s[3], 45);

	return result;
}

xoshiro256+ is about 15% faster than xoshiro256**, but the lowest three bits have lower linear complexity; therefore, only the upper 53 bits should be extracted and used for floating point results.

#include <stdint.h>

uint64_t rol64(uint64_t x, int k)
{
    
    
	return (x << k) | (x >> (64 - k));
}

struct xoshiro256p_state {
    
    
	uint64_t s[4];
};

uint64_t xoshiro256p(struct xoshiro256p_state *state)
{
    
    
	uint64_t (*s)[4] = &state->s;
	uint64_t const result = s[0] + s[3];
	uint64_t const t = s[1] << 17;

	s[2] ^= s[0];
	s[3] ^= s[1];
	s[1] ^= s[2];
	s[0] ^= s[3];

	s[2] ^= t;
	s[3] = rol64(s[3], 45);

	return result;
}

Note: If space is limited, xoroshiro128 ** is equivalent to xoshiro256 **, and xoroshiro128+ is equivalent to xoshiro256+. They have a small state space and are therefore not of much use for massively parallel programs. xoroshiro128+ also exhibits a slight dependence on Hamming weights, failing after outputting 5TB of data.
For 32-bit output, xoshiro128** and xoshiro128+ are exactly equivalent to xoshiro256** and xoshiro256+.

Algorithm testing

In principle, when designing a random algorithm, we would want the initial state to use as little memory as possible, execute faster, have a longer cycle, and have higher randomness quality.

Theoretical testing mainly tests the period, intrinsic structure and correlation of random numbers . However, theoretical testing is not enough, we also need empirical testing. Empirical testing is relatively simple, and there are many methods. For example: equal distribution test: also called uniformity test, divide the [0,1) interval into k equal sub-intervals, and the number of pseudo-random numbers falling in each sub-interval should be equal. The commonly used one is the x^2 test. Sequence test: Successive numbers are evenly independent and uniformly distributed. Counts the number of occurrences of even pairs. Interval test: used to examine the length of the interval between occurrences of a sequence within a certain range. Poker Test: Consider n groups of 5 consecutive integers and observe the pattern…

Quality testing of randomness can mainly be done through the tool TestU01 , which is a software library implemented in ANSI C and provides a collection of utilities for empirical statistical testing of uniform random number generators.
What TestU01 directly tests must be a [0,1)function that can generate floating point numbers or a function that can generate 32-bit fully random unsigned integers.
Its entire set of tests is called BigCrush and is currently the most stringent random number test suite. It can well detect the reliability of a random number generator.

As shown in the figure below, only 7 RNG algorithms passed the test. The detailed test report can be found at:
https://www.pcg-random.org/pdf/toms-oneill-pcg-family-v1.02.pdf

p-8

*There are many excellent PRNGs, such as the very safe and extremely long-period Mersenne Twiste (Mersenne Twiste algorithm), which this article will not record for the time being.

Implementation

1.ECMAScript（JavaScript）

The main way to generate random numbers in js is undoubtedly through Math.random(). First, let’s look at the introduction of this method in the official document ( ES6 ):

p-1

A simple translation is to generate numbers in 0～1a range randomly or pseudo-randomly with a roughly uniform distribution within the range. 0～1Taking the V8 engine as an example, let's take a look at Math.random()the specific implementation:

Until version v4.9.40, V8 chose the MWC1616 algorithm. The core content is:

uint32_t state0 = 1;
uint32_t state1 = 2;
uint32_t mwc1616 () {
  state0 = 18030 * (state0 & 0xFFFF) + (state0 >> 16);
  state1 = 30903 * (state1 & 0xFFFF) + (state1 >> 16);
  return state0 << 16 + (state1 & 0xFFFF);
}

The MWC1616's calculations are fast and use very little memory, but its quality is below par:

1. Range limitation: The random number range of this algorithm is 2^32within the range, but double-precision floating point numbers can represent 2^52decimals 0～1;
2. Result control: The upper part of the result value is very dependent on state0; if the initial state is not selected correctly, the cycle length may be less than 400 million;
3. Unable to pass multiple tests of TestU01.

Therefore, V8 has optimized the random number algorithm (starting with v4.9.41.0, starting with Chrome49):

The latest (2021.04.11) V8 implementation (directory address: v8 / src / numbers /)

First look at Math.random()the source code ( v8 / src / numbers / math-random.cc ):

// Copyright 2018 the V8 project authors. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.

#include "src/numbers/math-random.h"

#include "src/base/utils/random-number-generator.h"
#include "src/common/assert-scope.h"
#include "src/execution/isolate.h"
#include "src/objects/contexts-inl.h"
#include "src/objects/fixed-array.h"
#include "src/objects/smi.h"

namespace v8 {
namespace internal {

void MathRandom::InitializeContext(Isolate* isolate,
                                   Handle<Context> native_context) {
  Handle<FixedDoubleArray> cache = Handle<FixedDoubleArray>::cast(
      isolate->factory()->NewFixedDoubleArray(kCacheSize));
  for (int i = 0; i < kCacheSize; i++) cache->set(i, 0);
  native_context->set_math_random_cache(*cache);
  Handle<PodArray<State>> pod =
      PodArray<State>::New(isolate, 1, AllocationType::kOld);
  native_context->set_math_random_state(*pod);
  ResetContext(*native_context);
}

void MathRandom::ResetContext(Context native_context) {
  native_context.set_math_random_index(Smi::zero());
  State state = {0, 0};
  PodArray<State>::cast(native_context.math_random_state()).set(0, state);
}

Address MathRandom::RefillCache(Isolate* isolate, Address raw_native_context) {
  Context native_context = Context::cast(Object(raw_native_context));
  DisallowHeapAllocation no_gc;
  PodArray<State> pod =
      PodArray<State>::cast(native_context.math_random_state());
  State state = pod.get(0);
  // Initialize state if not yet initialized. If a fixed random seed was
  // requested, use it to reset our state the first time a script asks for
  // random numbers in this context. This ensures the script sees a consistent
  // sequence.
  if (state.s0 == 0 && state.s1 == 0) {
    uint64_t seed;
    if (FLAG_random_seed != 0) {
      seed = FLAG_random_seed;
    } else {
      isolate->random_number_generator()->NextBytes(&seed, sizeof(seed));
    }
    state.s0 = base::RandomNumberGenerator::MurmurHash3(seed);
    state.s1 = base::RandomNumberGenerator::MurmurHash3(~seed);
    CHECK(state.s0 != 0 || state.s1 != 0);
  }

  FixedDoubleArray cache =
      FixedDoubleArray::cast(native_context.math_random_cache());
  // Create random numbers.
  for (int i = 0; i < kCacheSize; i++) {
    // Generate random numbers using xorshift128+.
    base::RandomNumberGenerator::XorShift128(&state.s0, &state.s1);
    cache.set(i, base::RandomNumberGenerator::ToDouble(state.s0));
  }
  pod.set(0, state);

  Smi new_index = Smi::FromInt(kCacheSize);
  native_context.set_math_random_index(new_index);
  return new_index.ptr();
}

}  // namespace internal
}  // namespace v8

Overall, the main content of this file related to random number generation:

Generate and control random seeds sum state.s0( state.s1method called random-number-generator.h: base::RandomNumberGenerator::MurmurHash3(seed)sum base::RandomNumberGenerator::MurmurHash3(~seed))
- *Where seed comes from v8 / src / base / utils / random-number-generator.cc :
```
int64_t seed = Time::NowFromSystemTime().ToInternalValue() << 24;
  seed ^= TimeTicks::HighResolutionNow().ToInternalValue() << 16;
  seed ^= TimeTicks::Now().ToInternalValue() << 8;
  SetSeed(seed);
```
  That is, bit operations are performed based on time information, so seed is also a pseudo-random number.
random-number-generator.hMethod called :base::RandomNumberGenerator::XorShift128(&state.s0, &state.s1);
Set up and control seed caching

So the core generation is still in the v8 / src / base / utils / random-number-generator.cc file:

// Copyright 2013 the V8 project authors. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.

#include "src/base/utils/random-number-generator.h"

#include <stdio.h>
#include <stdlib.h>

#include <algorithm>
#include <new>

#include "src/base/bits.h"
#include "src/base/macros.h"
#include "src/base/platform/mutex.h"
#include "src/base/platform/time.h"
#include "src/base/platform/wrappers.h"

namespace v8 {
namespace base {

static LazyMutex entropy_mutex = LAZY_MUTEX_INITIALIZER;
static RandomNumberGenerator::EntropySource entropy_source = nullptr;

// static
void RandomNumberGenerator::SetEntropySource(EntropySource source) {
  MutexGuard lock_guard(entropy_mutex.Pointer());
  entropy_source = source;
}


RandomNumberGenerator::RandomNumberGenerator() {
  // Check if embedder supplied an entropy source.
  {
    MutexGuard lock_guard(entropy_mutex.Pointer());
    if (entropy_source != nullptr) {
      int64_t seed;
      if (entropy_source(reinterpret_cast<unsigned char*>(&seed),
                         sizeof(seed))) {
        SetSeed(seed);
        return;
      }
    }
  }

#if V8_OS_CYGWIN || V8_OS_WIN
  // Use rand_s() to gather entropy on Windows. See:
  // https://code.google.com/p/v8/issues/detail?id=2905
  unsigned first_half, second_half;
  errno_t result = rand_s(&first_half);
  DCHECK_EQ(0, result);
  result = rand_s(&second_half);
  DCHECK_EQ(0, result);
  SetSeed((static_cast<int64_t>(first_half) << 32) + second_half);
#elif V8_OS_MACOSX || V8_OS_FREEBSD || V8_OS_OPENBSD
  // Despite its prefix suggests it is not RC4 algorithm anymore.
  // It always succeeds while having decent performance and
  // no file descriptor involved.
  int64_t seed;
  arc4random_buf(&seed, sizeof(seed));
  SetSeed(seed);
#else
  // Gather entropy from /dev/urandom if available.
  FILE* fp = base::Fopen("/dev/urandom", "rb");
  if (fp != nullptr) {
    int64_t seed;
    size_t n = fread(&seed, sizeof(seed), 1, fp);
    base::Fclose(fp);
    if (n == 1) {
      SetSeed(seed);
      return;
    }
  }

  // We cannot assume that random() or rand() were seeded
  // properly, so instead of relying on random() or rand(),
  // we just seed our PRNG using timing data as fallback.
  // This is weak entropy, but it's sufficient, because
  // it is the responsibility of the embedder to install
  // an entropy source using v8::V8::SetEntropySource(),
  // which provides reasonable entropy, see:
  // https://code.google.com/p/v8/issues/detail?id=2905
  int64_t seed = Time::NowFromSystemTime().ToInternalValue() << 24;
  seed ^= TimeTicks::HighResolutionNow().ToInternalValue() << 16;
  seed ^= TimeTicks::Now().ToInternalValue() << 8;
  SetSeed(seed);
#endif  // V8_OS_CYGWIN || V8_OS_WIN
}


int RandomNumberGenerator::NextInt(int max) {
  DCHECK_LT(0, max);

  // Fast path if max is a power of 2.
  if (bits::IsPowerOfTwo(max)) {
    return static_cast<int>((max * static_cast<int64_t>(Next(31))) >> 31);
  }

  while (true) {
    int rnd = Next(31);
    int val = rnd % max;
    if (std::numeric_limits<int>::max() - (rnd - val) >= (max - 1)) {
      return val;
    }
  }
}


double RandomNumberGenerator::NextDouble() {
  XorShift128(&state0_, &state1_);
  return ToDouble(state0_);
}


int64_t RandomNumberGenerator::NextInt64() {
  XorShift128(&state0_, &state1_);
  return bit_cast<int64_t>(state0_ + state1_);
}


void RandomNumberGenerator::NextBytes(void* buffer, size_t buflen) {
  for (size_t n = 0; n < buflen; ++n) {
    static_cast<uint8_t*>(buffer)[n] = static_cast<uint8_t>(Next(8));
  }
}

static std::vector<uint64_t> ComplementSample(
    const std::unordered_set<uint64_t>& set, uint64_t max) {
  std::vector<uint64_t> result;
  result.reserve(max - set.size());
  for (uint64_t i = 0; i < max; i++) {
    if (!set.count(i)) {
      result.push_back(i);
    }
  }
  return result;
}

std::vector<uint64_t> RandomNumberGenerator::NextSample(uint64_t max,
                                                        size_t n) {
  CHECK_LE(n, max);

  if (n == 0) {
    return std::vector<uint64_t>();
  }

  // Choose to select or exclude, whatever needs fewer generator calls.
  size_t smaller_part = static_cast<size_t>(
      std::min(max - static_cast<uint64_t>(n), static_cast<uint64_t>(n)));
  std::unordered_set<uint64_t> selected;

  size_t counter = 0;
  while (selected.size() != smaller_part && counter / 3 < smaller_part) {
    uint64_t x = static_cast<uint64_t>(NextDouble() * max);
    CHECK_LT(x, max);

    selected.insert(x);
    counter++;
  }

  if (selected.size() == smaller_part) {
    if (smaller_part != n) {
      return ComplementSample(selected, max);
    }
    return std::vector<uint64_t>(selected.begin(), selected.end());
  }

  // Failed to select numbers in smaller_part * 3 steps, try different approach.
  return NextSampleSlow(max, n, selected);
}

std::vector<uint64_t> RandomNumberGenerator::NextSampleSlow(
    uint64_t max, size_t n, const std::unordered_set<uint64_t>& excluded) {
  CHECK_GE(max - excluded.size(), n);

  std::vector<uint64_t> result;
  result.reserve(max - excluded.size());

  for (uint64_t i = 0; i < max; i++) {
    if (!excluded.count(i)) {
      result.push_back(i);
    }
  }

  // Decrease result vector until it contains values to select or exclude,
  // whatever needs fewer generator calls.
  size_t larger_part = static_cast<size_t>(
      std::max(max - static_cast<uint64_t>(n), static_cast<uint64_t>(n)));

  // Excluded set may cause that initial result is already smaller than
  // larget_part.
  while (result.size() != larger_part && result.size() > n) {
    size_t x = static_cast<size_t>(NextDouble() * result.size());
    CHECK_LT(x, result.size());

    std::swap(result[x], result.back());
    result.pop_back();
  }

  if (result.size() != n) {
    return ComplementSample(
        std::unordered_set<uint64_t>(result.begin(), result.end()), max);
  }
  return result;
}

int RandomNumberGenerator::Next(int bits) {
  DCHECK_LT(0, bits);
  DCHECK_GE(32, bits);
  XorShift128(&state0_, &state1_);
  return static_cast<int>((state0_ + state1_) >> (64 - bits));
}


void RandomNumberGenerator::SetSeed(int64_t seed) {
  initial_seed_ = seed;
  state0_ = MurmurHash3(bit_cast<uint64_t>(seed));
  state1_ = MurmurHash3(~state0_);
  CHECK(state0_ != 0 || state1_ != 0);
}


uint64_t RandomNumberGenerator::MurmurHash3(uint64_t h) {
  h ^= h >> 33;
  h *= uint64_t{0xFF51AFD7ED558CCD};
  h ^= h >> 33;
  h *= uint64_t{0xC4CEB9FE1A85EC53};
  h ^= h >> 33;
  return h;
}

}  // namespace base
}  // namespace v8

It can be seen that the core is the XorShift128+ algorithm:

static inline void XorShift128(uint64_t* state0, uint64_t* state1) {
    uint64_t s1 = *state0;
    uint64_t s0 = *state1;
    *state0 = s0;
    s1 ^= s1 << 23;
    s1 ^= s1 >> 17;
    s1 ^= s0;
    s1 ^= s0 >> 26;
    *state1 = s1;
}

It uses 128-bit internal state, has a cycle length of 2^128 - 1, and XorShift128+ passes all tests of TestU01.

However, there will still be a danger of "cryptographically secure pseudo-random number generation" after the update, and for use cases such as hashing, signature generation, and encryption and decryption, ordinary PRNG is not suitable. So there is the following module.

crypto

In order to solve the above problems, browsers and node provide solutions:

1. Browser environment (webkit): Security variable API - crypto, window.crypto.getRandomValues

Compatibility: Overall compatibility is good

Mobile: Starting with iOS8, starting with Android5
PC: IE11, Chrome37 starting

p-6

2.nodejs environment: encryption module-crypto, crypto.randomFill

Compatibility: This method or alternative methods can also be seen in the v4.x documentation, so there are almost no compatibility issues.

The usage of these two methods is not introduced in this article. In order to ensure sufficient performance, they do not use a real random number generator, but use an entropy value with sufficient high quality as the seed of a pseudo-random number generator, such as an operating system. Entropy source (eg "/dev/urandom").

They are all methods of returning cryptographically secure random values at the expense of performance, and uuid uses them.

Webkit based browsers (Chrome, Safari) appear to use an ARC4 based RNG which discards early key streams and mixes system OS entropy every 1.5mb.

webkit encryption module code ( /Source/WebCore/page/Crypto.cpp):

/*
 * Copyright (C) 2011 Google Inc. All rights reserved.
 * Copyright (C) 2013 Apple Inc. All rights reserved.
 *
 * Redistribution and use in source and binary forms, with or without
 * modification, are permitted provided that the following conditions
 * are met:
 *
 * 1.  Redistributions of source code must retain the above copyright
 *     notice, this list of conditions and the following disclaimer. 
 * 2.  Redistributions in binary form must reproduce the above copyright
 *     notice, this list of conditions and the following disclaimer in the
 *     documentation and/or other materials provided with the distribution. 
 * 3.  Neither the name of Google, Inc. ("Google") nor the names of
 *     its contributors may be used to endorse or promote products derived
 *     from this software without specific prior written permission. 
 *
 * THIS SOFTWARE IS PROVIDED BY GOOGLE AND ITS CONTRIBUTORS "AS IS" AND ANY
 * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
 * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
 * DISCLAIMED. IN NO EVENT SHALL APPLE OR ITS CONTRIBUTORS BE LIABLE FOR ANY
 * DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
 * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
 * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
 * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
 * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
 * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 */


#include "config.h"
#include "Crypto.h"

#include "Document.h"
#include "SubtleCrypto.h"
#include <JavaScriptCore/ArrayBufferView.h>
#include <wtf/CryptographicallyRandomNumber.h>

#if OS(DARWIN)
#include <CommonCrypto/CommonCryptor.h>
#include <CommonCrypto/CommonRandom.h>
#endif

namespace WebCore {

Crypto::Crypto(ScriptExecutionContext* context)
    : ContextDestructionObserver(context)
#if ENABLE(WEB_CRYPTO)
    , m_subtle(SubtleCrypto::create(context))
#endif
{
}

Crypto::~Crypto() = default;

ExceptionOr<void> Crypto::getRandomValues(ArrayBufferView& array)
{
    if (!isInt(array.getType()))
        return Exception { TypeMismatchError };
    if (array.byteLength() > 65536)
        return Exception { QuotaExceededError };
#if OS(DARWIN)
    auto rc = CCRandomGenerateBytes(array.baseAddress(), array.byteLength());
    RELEASE_ASSERT(rc == kCCSuccess);
#else
    cryptographicallyRandomValues(array.baseAddress(), array.byteLength());
#endif
    return { };
}

#if ENABLE(WEB_CRYPTO)

SubtleCrypto& Crypto::subtle()
{
    return m_subtle;
}

#endif

}

It can be seen that the core is the method wtf/CryptographicallyRandomNumber.hcalled cryptographicallyRandomValues.

The encryption implementation can be seen at Source/WTF/wtf/CryptographicallyRandomNumber.cpp:


/*
 * Copyright (c) 1996, David Mazieres <[email protected]>
 * Copyright (c) 2008, Damien Miller <[email protected]>
 *
 * Permission to use, copy, modify, and distribute this software for any
 * purpose with or without fee is hereby granted, provided that the above
 * copyright notice and this permission notice appear in all copies.
 *
 * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
 * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
 * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
 * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
 * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
 * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
 * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
 */

/*
 * Arc4 random number generator for OpenBSD.
 *
 * This code is derived from section 17.1 of Applied Cryptography,
 * second edition, which describes a stream cipher allegedly
 * compatible with RSA Labs "RC4" cipher (the actual description of
 * which is a trade secret).  The same algorithm is used as a stream
 * cipher called "arcfour" in Tatu Ylonen's ssh package.
 *
 * RC4 is a registered trademark of RSA Laboratories.
 */

#include "config.h"
#include <wtf/CryptographicallyRandomNumber.h>

#include <mutex>
#include <wtf/Lock.h>
#include <wtf/NeverDestroyed.h>
#include <wtf/OSRandomSource.h>

namespace WTF {

namespace {

class ARC4Stream {
    WTF_MAKE_FAST_ALLOCATED;
public:
    ARC4Stream();

    uint8_t i;
    uint8_t j;
    uint8_t s[256];
};

class ARC4RandomNumberGenerator {
    WTF_MAKE_FAST_ALLOCATED;
public:
    ARC4RandomNumberGenerator();

    uint32_t randomNumber();
    void randomValues(void* buffer, size_t length);

private:
    inline void addRandomData(unsigned char *data, int length);
    void stir();
    void stirIfNeeded();
    inline uint8_t getByte();
    inline uint32_t getWord();

    ARC4Stream m_stream;
    int m_count;
    Lock m_mutex;
};

ARC4Stream::ARC4Stream()
{
    for (int n = 0; n < 256; n++)
        s[n] = n;
    i = 0;
    j = 0;
}

ARC4RandomNumberGenerator::ARC4RandomNumberGenerator()
    : m_count(0)
{
}

void ARC4RandomNumberGenerator::addRandomData(unsigned char* data, int length)
{
    m_stream.i--;
    for (int n = 0; n < 256; n++) {
        m_stream.i++;
        uint8_t si = m_stream.s[m_stream.i];
        m_stream.j += si + data[n % length];
        m_stream.s[m_stream.i] = m_stream.s[m_stream.j];
        m_stream.s[m_stream.j] = si;
    }
    m_stream.j = m_stream.i;
}

void ARC4RandomNumberGenerator::stir()
{
    unsigned char randomness[128];
    size_t length = sizeof(randomness);
    cryptographicallyRandomValuesFromOS(randomness, length);
    addRandomData(randomness, length);

    // Discard early keystream, as per recommendations in:
    // http://www.wisdom.weizmann.ac.il/~itsik/RC4/Papers/Rc4_ksa.ps
    for (int i = 0; i < 256; i++)
        getByte();
    m_count = 1600000;
}

void ARC4RandomNumberGenerator::stirIfNeeded()
{
    if (m_count <= 0)
        stir();
}

uint8_t ARC4RandomNumberGenerator::getByte()
{
    m_stream.i++;
    uint8_t si = m_stream.s[m_stream.i];
    m_stream.j += si;
    uint8_t sj = m_stream.s[m_stream.j];
    m_stream.s[m_stream.i] = sj;
    m_stream.s[m_stream.j] = si;
    return (m_stream.s[(si + sj) & 0xff]);
}

uint32_t ARC4RandomNumberGenerator::getWord()
{
    uint32_t val;
    val = getByte() << 24;
    val |= getByte() << 16;
    val |= getByte() << 8;
    val |= getByte();
    return val;
}

uint32_t ARC4RandomNumberGenerator::randomNumber()
{
    auto locker = holdLock(m_mutex);

    m_count -= 4;
    stirIfNeeded();
    return getWord();
}

void ARC4RandomNumberGenerator::randomValues(void* buffer, size_t length)
{
    auto locker = holdLock(m_mutex);

    unsigned char* result = reinterpret_cast<unsigned char*>(buffer);
    stirIfNeeded();
    while (length--) {
        m_count--;
        stirIfNeeded();
        result[length] = getByte();
    }
}

ARC4RandomNumberGenerator& sharedRandomNumberGenerator()
{
    static LazyNeverDestroyed<ARC4RandomNumberGenerator> randomNumberGenerator;
    static std::once_flag onceFlag;
    std::call_once(
        onceFlag,
        [] {
            randomNumberGenerator.construct();
        });

    return randomNumberGenerator;
}

}

uint32_t cryptographicallyRandomNumber()
{
    return sharedRandomNumberGenerator().randomNumber();
}

void cryptographicallyRandomValues(void* buffer, size_t length)
{
    sharedRandomNumberGenerator().randomValues(buffer, length);
}

}

In comparison Math.random(), it can be seen that the implementation of crypto is more complex and depends on os.

The author also discovered that webkit, such as WebRtc, implements random algorithms separately, and its complexity is between crypto and Math.random(). If you are interested, you can take a look at the document /Source/webrtc/rtc_base/helpers.cc.

Later, we casually use js to write several js implementations of the previous PRNG (although it has almost no practical significance):

Simple implementation of random numbers

Method 1: MWC1616

const Utils = {
    
    
  state0: 1,
  state1: 2,
  getRandom: () => {
    
    
    let state0 = Utils.state0;
    let state1 = Utils.state1;
    Utils.state0 = state0 = 18030 * (state0 & 0xffff) + (state0 >> 16);
    Utils.state1 = state1 = 30903 * (state1 & 0xffff) + (state1 >> 16);
    return state0 << 16 + (state1 & 0xffff);
  }
};

// use:
// Utils.getRandom();

Method 2: XorShift

npm-xorshift

const Utils = {
    
    
    t: undefined,
    x: 123456789, 
    y: 362436069, 
    z: 521288629,
    getRandom () {
    
    
        let x = Utils.x;
        x ^= x << 16;
        x ^= x >> 5;
        x ^= x << 1;
    
        Utils.t = x;
        Utils.x = Utils.y;
        Utils.y = Utils.z;
        Utils.z = Utils.t ^ Utils.x ^ Utils.y;
        return Utils.z;
    }
}

Method 3: XorShift128+

https://www.npmjs.com/package/xorshift128plus

const Utils = {
    
    
  state0: 1,
  state1: 2,
  getRandom () {
    
    
    let s1 = Utils.state0;
    let s0 = Utils.state1;
    Utils.state0 = s0;
    s1 ^= s1 << 23;
    s1 ^= s1 >> 17;
    s1 ^= s0;
    s1 ^= s0 >> 26;
    Utils.state1 = s1;
    return Utils.state0 + Utils.state1;
  }
}

at last

This article is quite long, so let’s summarize the main contents:

1. Introduction, function and definition of random numbers/random sequences;
2. Introduction to some common random algorithms and detection tools;
3. Algorithm improvement of V8 engine
4. js implementation of random algorithm

Finally, introduce a famous saying:

Any one who considers arithmetical methods of producing random digits is , of course, in a state of sin. ——JOHN VON NEUMANN, 1951

Article blog address: http://blog.michealwayne.cn/2021/04/11/notes/%E9%9A%8F%E6%9C%BA%E6%95%B0%E4%B8%8E%E8% AE%A1%E7%AE%97%E6%9C%BA%E5%AE%9E%E7%8E%B0/If you
have any suggestions or reprints -> [email protected]