In a distributed system, even a small cluster needs a mechanism to maintain the state consistency and data consistency of each node or service. zab and paxos use earlier protocols, and later the RAFT protocol appeared. These are mainly connected to nodes through the RPC mechanism, so they are mainly used in local area networks or data centers. Gossip (meaning: gossip, gossip) supports the p2p protocol, which is a protocol that can be used for the consistency maintenance of WAN, and has been adopted by many Internet systems. In addition, CRDT is also a consensus mechanism under research, which can realize instant multi-node data write synchronization. Here we mainly introduce the GOSSIP protocol.
refer to:
- Raft Consensus Protocol, https://my.oschina.net/u/2306127/blog/1617192
- Raft Algorithm and Gossip Protocol, https://www.backendcloud.cn/2017/11/12/raft-gossip/
- Cassandra's gossip protocol, http://blog.csdn.net/aesop_wubo/article/details/20401431
Gossip is a widely used protocol in distributed systems, which is mainly used to realize information exchange between distributed nodes or processes. The Gossip protocol also meets the requirements of low load, high reliability and scalability required by the application layer multicast protocol. Due to its simplicity and ease of implementation, many current systems (such as Amazon S3, Usenet NNTP, etc.) choose to implement the application-layer multicast function based on the Gossip protocol.
What is the Gossip Protocol
Gossip Protocol utilizes a random way to spread information across the network. Just as the meaning of Gossip itself, the workflow of the Gossip protocol is similar to the spread of gossip, or the spread of epidemics. Specifically, the Gossip Protocol can be divided into two types: Push-based and Pull-based. The specific workflow of Push-based Gossip Protocol is as follows:
- A node in the network randomly selects other bb nodes as transmission objects.
- The node transmits corresponding information to its selected bb nodes
- The node that receives the information repeats the same work
The protocol process of Pull-based Gossip Protocol is just the opposite:
- A node vv randomly selects bb nodes to ask if there is the latest information
- The node that received the request replies to the node vv with information that it has not received recently
Of course, in order to improve the performance of the Gossip protocol, there is also a hybrid protocol based on Push-Pull. At the same time, it should be noted that the Gossip protocol does not make any requirements for the quality of the network, nor does it require a loss-free network protocol. The Gossip protocol is generally implemented based on UDP, which itself ensures the robustness of the protocol at the application layer.
Performance of the Gossip Protocol
The analysis of the Gossip protocol is based on epidemiological studies. Therefore, before analyzing the performance of Gossip, it is necessary to first introduce the basic model in epidemiology.
Epidemiology
The most basic model of epidemic contagion only makes the following assumptions:
- (n+1)(n+1) individuals are evenly distributed together
- The infection probability between each pair of populations is ββ, obviously 0<β<10<β<1.
- At any time, a person is either in an infected state or an uninfected state.
- Once a person transitions from the uninfected state to the infected state, it stays in the infected state.
With the above assumptions, we can further analyze the contagion of epidemics. We record the number of people who are in the infected state at tt as ytyt, and the people in the uninfected state as xtxt, then the initial state y0=1y0=1, x0=nx0=n, and at any time xt+yt=n+1xt+yt=n+ 1.
Considering continuous time, we know:
dxdt=−βxydxdt=−βxy
Solved:
x = n (n + 1) n + eβ (n + 1) tx = n (n + 1) n + eβ (n + 1) ty = n + 11 + ne − β (n + 1) ty = n + 11 + ne − β (n + 1) t
Obviously, when t→∞t→∞, x→0, y→(n+1) x→0, y→(n+1), that is, after enough time, all people will be infected.
Gossip performance
The above epidemic contagion model provides the basis for analyzing the performance of Gossip. In Gossip performance, we can consider: β=b/nβ=b/n (because for each node, the probability of being selected by other nodes is b/nb/n). We make t=clog(n)t=clog(n), we can get:
y≈(n+1)−1ncb−2y≈(n+1)−1ncb−2
This shows that it only takes O(log(n))O(log(n)) rounds for the gossip protocol to deliver information to all nodes. According to the analysis, the Gossip protocol has the following characteristics:
- Low latency. Only a transit time of O(log(n))O(log(n)) rounds is required.
- Very reliable. Only 1ncb−21ncb−2 nodes will not receive information.
- Lightweight. Each node transmits cblog(n) cblog(n) times.
For this colleague, the fault tolerance of the Gossip protocol is relatively high. For example, the packet loss rate of 5050 is equivalent to using b/2b/2 band instead of bb for analysis; the node error of 5050 is equivalent to using n/2n/2 instead of nn , and use b/2b/2 instead of bb for analysis, and the analysis results do not bring about an order of magnitude change.