Deployment of NameServer
Regarding NameServer, our previous article has explained the content of clustering in detail, and here we directly deploy it to three machines as a high-availability cluster
Broker deployment
We have also mentioned the deployment of Broker before, mainly using Dledger after version 4.5 to automatically switch the master-slave cluster
What is the communication protocol between Broker and NameServer? http, rpc or tcp?
In fact, they use TCP long connection communication, that is to say, Broker will establish a TCP long connection with each NameServer, and then send heartbeat requests through the TCP long connection at regular intervals.
Deployment of systems (producers and consumers) that access MQ
There must be a large number of systems accessing RocketMQ, because RocketMQ was born for this purpose, and some systems themselves are both producers and consumers, so the deployment of these systems should also be taken into consideration.
The consideration of the deployment of these systems should not be considered by the MQ department. If the system itself is owned by the company, some suggestions can be made to allow producers and consumers to be deployed in clusters to ensure high availability. But if it is a third-party system, we cannot intervene. All we can do is to consider how to make MQ operate normally when the third-party system crashes and cannot communicate with MQ normally.
What is Topic
Topic is the core data model of mq. If the direct translation is the meaning of topic, but when you hear the explanation of topic, do you feel confused? Do you immediately think of mobile phone topics and computer topics?
Therefore, it cannot be translated literally. It expresses the meaning of a data collection . The collection is the same type of data, and different types of data are stored in different topics.
Therefore, no matter whether the system wants to write messages or read data, it first needs to define the topic first, and then obtain the same type of data from the defined topic.
So how is Topic stored in Broker?
The way of storage is actually distributed storage. When we define a Topic, we specify that the data in it is distributed to multiple Brokers for storage. One thing to note here is that the object of distribution is actually the MasterBroker, and the SlaveBroker will pull data from the MasterBroker and exist as a copy. When the Broker sends a heartbeat to the NameServer, it will tell the NameServer which Broker the Topic is stored in.
How producers send messages to Broker
As we talked earlier, before sending a message, you first define a topic, and then when sending a message, you need to specify which topic you want to send it to.
Now that we know which topic we want to send to, the next step is to locate the location of the topic, how to locate it? It is to establish a long Tcp connection with the NameServer, pull the registration information regularly, and obtain which Brokers the Topic is currently assigned to. Then you can select a Broker according to the load balancing algorithm (the specific load balancing algorithm will be introduced later in the article).
After selecting the Broker, you can establish a Tcp persistent connection with the Broker, and send a message to the Topic in the Broker through the Tcp persistent connection.
After the Broker receives the message, it will store the message in the disk, and then the SlaveBroker and MasterBroker data will be synchronized to form a copy to ensure high availability.
The whole process is like this.
How consumers consume messages from Broker
After talking about the process of producers sending messages, let's talk about the process of consumers consuming messages.
In fact, the process of consumers consuming messages is similar to that of producers. The first step is to define Topics, then obtain information from NameServer, locate multiple Brokers where Topics are located, and then load balance and locate the Brokers to be accessed, and establish with Brokers. Connect to get messages.
The only difference here is that when getting the news again, it may be obtained on the MasterBroker, or it may be obtained on the SlaveBroker, depending on the situation at the time.
Overall Architecture Summary
Finally, let's take a look at this architecture, which can achieve complete high availability.
Clustered deployment of NameServer, clustered deployment of Broker, master-slave can also be automatically switched through Dledger, producers and consumers are also deployed in clusters, and it will not be affected if one is hung up.
Moreover, this architecture is not afraid of high concurrency. Messages under high concurrency can be distributed to multiple Brokers for processing, reducing system pressure.
Then our cluster can store massive messages because the storage method is distributed.
Finally, this architecture is scalable. If the concurrency of business requirements increases, the number of Brokers can also be expanded to support higher concurrency and larger storage.
In this way, our RocketMQ production deployment architecture is complete.
Well, that's all for today, welcome friends to enter the world of message middleware together.
at last
The editor is here to share some of my usual learning materials. Due to space limitations, the detailed information in the pdf document is too comprehensive and there are too many details, so I only take screenshots of some knowledge points for a rough introduction. Each small node contains There are more detailed content! Programmers (Yuan) in need can follow + like, click here to get it for free
Programmer code interview guide IT famous enterprise algorithm and data structure topic optimal solution
This is "this book of programmer interviews!" The book summarizes the optimal solutions to various questions in code interviews of famous IT companies, and provides relevant code implementations. Aiming at the pain point of lack of authoritative questions in current programmer interviews, this book The book selects nearly 200 classic code interview questions that have actually appeared, helping the majority of programmers to be foolproof in their interview preparation. After "scanning" this book, you are the "Question King"!
"TCP-IP Protocol Suite (4th Edition)"
This book is the latest version of the classic books that introduce the TCP/IP protocol family. Since the publication of the first edition, this book has been widely welcomed by readers.
The latest edition of this book is based on the latest development of computer network technology. The book has seven parts, a total of 30 chapters and seven appendices: the first part introduces some basic concepts and basic underlying technologies; the second part introduces network Layer protocols: the third part introduces the transport layer protocol; the fourth part introduces the application layer protocol; the fifth part introduces the next generation protocol, namely IPv6 protocol; the sixth part introduces network security issues; the seventh part gives 7 appendices.
Java Development Manual (Songshan Edition)
Needless to say, I will read Ali's development manual every time it is updated. This is the latest update in early August ** (Songshan version)**
MySQL 8 from entry to proficiency
The main content of this book includes MySQL installation and configuration, database creation, data table creation, data types and operators, MySQL functions, query data, data table operations (insert, update and delete data), indexes, stored procedures and Functions, views, triggers, user management, data backup and restoration, MySQL logs, performance optimization, MySQL Replication, MySQL Workbench, MySQL Utilities, MySQL Proxy, PHP operation MySQL database and PDO database abstract class library, etc. Finally, through the database design of 3 comprehensive cases, the application of MySQL in actual work is progressively described.
Spring5 Advanced Programming (5th Edition)
Covering everything in Spring 5, this book is the most comprehensive Spring reference and practical guide if you want to take full advantage of the power of this leading enterprise Java application development framework.
The fifth edition of this book covers core Spring and its integration with other leading Java technologies such as Hibemate JPA 2. Tls, Thymeleaf, and WebSocket. The focus of this book is on how to use Java configuration classes, lambda expressions, Spring Boot, and reactive programming. At the same time, some insights and real-world experiences will be shared with enterprise-level application developers, including remoting, transactions, Web and presentation layers, and more.
JAVA core knowledge points + 1000 Internet Java engineer interview questions
The Way of Enterprise IT Architecture Transformation Alibaba's Middle-Taiwan Strategic Thought and Architecture Actual Combat
This book tells the history of Alibaba's technology development, and it is also the history of practice and development of the Internet technology architecture.
[External link image transfer...(img-SiLUCck4-1625432911541)]
The Way of Enterprise IT Architecture Transformation Alibaba's Middle-Taiwan Strategic Thought and Architecture Actual Combat
This book tells the history of Alibaba's technology development, and it is also the history of practice and development of the Internet technology architecture.
