RocketMQ DLedger multiple copy that is the main switching from column contains a total of nine articles, the time span of around two months time, I think Give a man a fish than giving the fishing, whereby this series to show the series of creation the whole story, to show the author read the source code skills.
Before deciding in first reading multiple copies RocketMQ DLedger source (master-slave switch), and above all, to Baidu and other ways to make some basic understanding shared by the official, the function.
We learned RocketMQ provide primary before 4.5.0 from synchronization, that is, when the master node goes down, the end consumer can continue, but can not continue to send a message to the node from the replication group from the consumption of news. RocketMQ 4.5.0 version introduces multi-copy mechanism, namely DLedger, support from the main switch, that is, when the master node goes down in a replication group, the main trigger re-election in the replication group, you can continue after the completion of the primary election provide information write function. It is also known to switch from master rocketmq raft is based protocol.
raft agreement is the sort of person, I guess most readers of this term is not new, but like I just heard it roughly the same effect but not detailed study should be few, so I think that is looking RocketMQ DLedger multiple copies from the master We should focus on understanding the raft before switching protocol.
1, RocketMQ multiple copies pre articles: Preliminary agreement raft
In this paper, according to the official animation raft to raft learn about protocols, from basic paper that the raft main agreement contains two important parts: the primary election and log replication. After a raft understand the agreement primary election, basically copied the log, and then you can walk to RocketMQ DLedger multiple copies study that is the main source from switching, and to explore how big God is realized raft protocol. At the same time learned after the election the main part of raft of the agreement, it would be easy to think that if he raft to achieve agreement, which should be the key point to realize, when I think about the following:
2, source code analysis of multiple copies of the selected main RocketMQ DLedger Leader
According to an article on the idea, focusing on the realization DLedgerLeaderElector carried out a detailed analysis, in particular its internal state machine flow, and finally gives a flow chart of the primary election process and sort out a simple summary.
Tips: If you can not understand a moment in the process of reading the source code, you can allow it provides unit testing, DEBUG look, you can see the Clouds play a fog effect.
3, source code analysis RocketMQ DLedger multiple copy storage implementation
After studying in the primary election DLedger achieve the next breakthrough will focus on another part of the raft of agreements: Log copy. Because the log copy will involve the storage, so before learning log copy, first look at the DLedger storage-related design, such as the relationship between memory storage protocol DLedger log entries, log server organization, this classification other than RocketMQ commitlog .
4, source code analysis RocketMQ DLedger (multiple copies) of the log additional flow
In After completing multiple copies DLedger ie after switching from the main log storage, we will officially enter into the log copying part, from the graph we can easily understand, log replication actually consists of two large phases, the first refers to the master node ( after Leader) receiving client requests, the data is first stored in the main server, which then forwards the data to all the nodes. It concerns the first phase of this article: Log appended.
5, source code analysis RocketMQ DLedger (multi-copy) copy of the log (spread) paper copy of the log to continue to focus on the second phase, the master node that contains log forwarding, log arbitration node receives from the master node to log forwarding, namely the need to achieve only more than half of the cluster nodes are stored success was attributed to the message has been successfully submitted, it will be sent to the success of the click message client.
6, RocketMQ DLedger multiple copies log raft-based replication protocol design principles
Interpretation of the copied portion of source log raft protocol after more boring, so that the three carding flowchart herein, and points to achieve log to make a summary, in order to introduce multiple copies interpret rocketmq Dledger i.e. raft protocol switching from the main portion.
The previous six articles have focused on the primary election and log raft agreement replication. From the beginning of this section describes the implementation details handover, RAFT protocol has achieved the master node is selected from primary and log based replication is synchronous rocketmq primary master from another core switching master switch from a meta data, e.g. Topic, consumption set subscription information, news and other consumer progress. Further from the switching master is rocketmq 4.5.0 version only introduced if upgrading from an old version to 4.5.0, the original message is directly compatible among the heavy, so this article will be a detailed analysis of its design point.
8, source code analysis RocketMQ DLedger multi-master copy that is achieved from switching principle
Understand the smooth upgrade of skills from the design herein, this analysis will implement the main points from the switch, that is the focus of attention (especially synchronous message consumer progress) synchronous metadata from the source point of view.
. 9, RocketMQ DLedger i.e. multiple copies from the switching master combat
After foreshadowing the previous eight articles, I believe we have a new understanding of the realization of the principle of DLedger herein, this as a wrap up the series, describes how to upgrade from a master-slave synchronization cluster to DLedger smooth, that is, from the main switch version, and function verification.
What is the overall summary of the core elements of the first to recognize it as a whole, and then gradually, gradually decompose to form a an article, in the face can not read, can debug unit test cases provided by the official.
If this article to be helpful, then, to help spot trouble a praise, thank you.
Tips: This column is "RocketMQ technology insider" of the effort to build another wonderful series, but also "RocketMQ Technology Insider," the second edition of the original material.
About the author: "RocketMQ Technology Insider" author, RocketMQ community preacher, to safeguard the public number: middleware interested circles, you can scan the following QR code to interact with the author.