Season Class Kaihu PDFBinder supports opening encrypted PDF format, editing, copying and printing.

Concurrency means that multiple tasks can be performed at the same time. Concurrent programming has a broad meaning, including multi-threaded programming, multi-process programming, and distributed programs. The concurrency meaning explained in this chapter belongs to multithreaded programming.

Goroutine is scheduled by the Go runtime, while threads are scheduled by the operating system.

The user allocates enough tasks, the system can automatically help the user to assign tasks to the CPU, so that these tasks operate as concurrently as possible. This mechanism is called goroutine in Go language.

The Go program starts from the main () function in the main package. When the program starts, the Go program creates a default goroutine for the main () function.

Adjust concurrent running performance (DOMAXPROCS)

In traditional logic, developers need to maintain the correspondence between threads in the thread pool and the number of CPU cores. Similarly, Go can also be done through the runtime.GOMAXPROCS () function, the format is:

In common distributed systems, situations such as machine downtime or network anomalies (including delay, loss, duplication, out-of-sequence of messages, and network partitions) (that is, distributed systems where anomalies occur) will always occur. The problem that the Paxos algorithm needs to solve is how to quickly and correctly agree on the value of a certain data within the cluster in a distributed system where the above exceptions may occur. It can also be understood as the consistency of states achieved in a distributed system.

The core idea of ​​Base theory is final consistency. Even if strong consistency (Strong Consistency) cannot be achieved, each application can use appropriate methods to achieve eventual consistency (Eventual Consistency) according to its own business characteristics.

Quorum mechanism

Before understanding Paxos, let ’s first understand the "Quorum election algorithm". When you see the election algorithm, do you think of the leader election in zookeeper? Your intuition is okay.

The Quorum mechanism can be seen in various consistency algorithms. The main mathematical idea comes from the drawer principle. In one sentence, it is explained that in N copies, if there are W successful updates at one time, then I am reading When fetching data, it is necessary to read from more than N-W copies, so that at least one updated data can be read.

Corresponding to the Quorum mechanism is WARO, which is WriteAllReadone, is a simple copy control protocol. When the Client requests to write data to a certain copy (update data), this write operation is only after all the copies have been updated successfully. It is considered a success, otherwise it is regarded as a failure.

WARO gives priority to guarantee read service, because all copies are updated successfully, it can be regarded as a successful update, thus ensuring that all copies are consistent, in this case, only need to read the data on any one copy. The availability of the write service is low, because as long as one copy fails to update, the write operation is considered to have failed. Suppose there are N copies, and N-1 are all down. The remaining copy can still provide read service; but as long as one copy is down, the write service will not succeed.

WARO sacrifices the availability of update services and maximizes the availability of read services, and Quorum is a compromise between update services and read services.

Quorum definition

The definition of Quorum is as follows: Suppose there are N copies, and the update operation wi is successfully updated in W copies, the update operation wi is considered successful, and the data corresponding to the update operation successfully submitted this time is called: "Successfully submitted data ". For read operations, at least R copies are required to read the updated data. Among them, W + R> N, that is, W and R overlap, generally, W + R = N + 1.

N = number of stored data copies

W = copy required for successful update

R = number of copies to be accessed in one data object read

Quorum is limited to the need to read at least N + 1-w copies of data, which sounds abstract.

For example, we maintain 10 copies, and successfully update three at a time, then at least eight copies of data need to be read, which can ensure that we have read the latest data.

Quorum application

The Quorum mechanism cannot guarantee strong consistency, which means that any user or node can read the most recently successfully submitted copy data at any time.

The use of the Quorum mechanism needs to cooperate with a metadata service that obtains the latest successfully submitted version number, so that the latest version number that has been successfully submitted can be determined, and then the latest written data can be confirmed from the data that has been read.

Quorum is a mechanism commonly used in distributed systems. It is a voting algorithm used to ensure data redundancy and eventual consistency. The application of the Quorum mechanism can be seen in algorithms such as Paxos, Raft, and ZooKeeper's Zab.

Related concepts of Paxos algorithm

In the Paxos protocol, there are three types of node roles, namely Proposer, Acceptor and Learner, and there is also a Client as the issuer. Technical pictures

The above three types of roles are only logically divided. In work practice, a node can serve these three types of roles at the same time.

Proposer

There can be multiple Proposers. At the beginning of the process, Proposer proposes a motion, which is value. The so-called value can be any operation in the project, such as "modifying the value of a variable to a new value." These operations are abstracted as value.

Different Proposers can propose different or even contradictory values. For example, a Proposer proposes to "set variable X to 1" and another Proposer proposes to "set variable X to 2", but for the same round of Paxos process, there is only one value at most. It was approved.

Acceptor

In the cluster, there are N Acceptors, and the Acceptors are completely equal and independent. The value proposed by Proposer must be approved by more than half (N / 2 + 1) Acceptor before it can be passed.

Learner

Learner does not participate in the election, but learns the approved value. In Paxos, Learner mainly participates in the related state machine synchronization process.

Here Leaner's process refers to the Quorum council mechanism. A value needs to be approved by Acceptor with W = N / 2 + 1. Learner needs to read at least N / 2 + 1 Accpetors and after reading the results of N Acceptor at most. In order to learn a passed value. for example:

I have 10 Accpetors, read at least 6 identical values, and at most 10 identical values. That is, the result with the most votes is selected.

Client Issuer

The role of Client, as the issuer, does not actually participate in the election process, such as the source of the modification request. Interaction between Proposer and Acceptor

Interaction between Proposer and Acceptor

In Paxos, the proposal is submitted to the Acceptor, and the Acceptor agrees to select the final value, and then tells Learner the final value.

The interaction between Proposer and Acceptor mainly has 4 types of message communication, as shown below: