0 Github
1 face questions
How to ensure high availability of the message queue
2 test sites analysis
High availability is a must ask, because many shortcomings MQ, leading to reduced system availability.
So long as you use the MQ, the next is sure to ask some of the points around those shortcomings MQ of how to solve.
If you silly it dry with a MQ, various problems have never thought about it, then you cup, the interviewer the impression of you is that only some of the technology is simple and practical, without any thought, immediately your impression not too good.
If such students to recruit them to be ordinary within 20k salary okay brother. If you recruit them do pay more than 20 k of Engineering, it was tragic, it allows you to design a system, which certainly pile pit, out of the accident the company affected by the loss, the team pot back together.
3 Detailed face questions
The question you ask is good, because you can not ask how to ensure high availability kafka ah? ActiveMQ high availability guarantee how ah? If the interviewer asks a so it is not very level, people may use is RabbitMQ, not used Kafka, you ask people kafka up doing? This is not made clear what people make things difficult.
So there are levels of interviewer ask is how MQ high availability guarantee? This is what you used MQ, you talk about your understanding of the high availability of the MQ.
3.1 RabbitMQ High Availability
RabbitMQ is more representative, because it is based on the principal to explain how to achieve the first MQ made from high availability, we have to follow his example of high availability.
rabbitmq has three modes: stand-alone mode, normal mode cluster, the cluster mirroring mode
3.1.1 stand-alone mode
Is the demo level, the general is that you start a local children's play, with no one to produce stand-alone mode
3.1.2 ordinary cluster model
Start plurality RabbitMQ instances on multiple machines, each machine a start
But the queue you created, and only on a RabbitMQ instance, but each instance metadata synchronization queue. Over time you consume, in fact, if the connection to another instance, then that instance will pull examples from the queue where the data come on.
This approach does a lot of trouble, not how good, did not do so-called distributed, is a common cluster.
Because it leads to
- Consumers either randomly each time a connection instance and pull data
has pulled the overhead data - Either a fixed connection that queue instance where consumption data
performance bottlenecks single instance of
And if that's put down the queue instance, will lead to other instances of the following will not be able to pull from that instance, if you open the message persistence, let RabbitMQ floor to store the message, then the message will not necessarily lose, you have to wait for this example restored before you can continue to pull data from the queue.
So this thing is more awkward, and this is no so-called high availability at all, this program is mainly to improve the throughput, that allow multiple nodes in a cluster to serve a queue of read and write operations.
- Architecture as shown in FIG.
3.1.3 Mirror cluster model
This is the so-called RabbitMQ high availability mode, with ordinary cluster model is not the same, you create a queue, whether or metadata in the message queue will exist on more than one instance, then every time you write a message to the queue when automatically queue the message in the plurality of instances for message synchronization.
benefit
Any machine goes down, all right, other machines can be used
harm
- Performance overhead is too big now, synchronized news all the machines, resulting in network bandwidth consumption and heavy pressure!
- So play, there is no extension at all, and if a queue is heavily loaded, you add a machine, the new machine also contains all the data of the queue, and there is no way to expand your queue linear
So how to open this image cluster model? RabbitMQ is actually very simple and very good management console is a new strategy in the background, this strategy is a strategy mirrored cluster mode, the specified time may be required to synchronize data to all nodes, you can also ask to be synchronized to a specified number of node, and then you create the queue again when applying this strategy, it will automatically synchronize data to other nodes go up.
- Architecture as shown in FIG.
3.2 kafka high availability
kafka a basic understanding of architecture: multiple broker, each broker is a node; you create a topic, this topic can be divided into multiple partition, each partition can exist on a different broker, put each partition part of the data.
This is a natural distributed message queue, that is a topic data is dispersed on multiple machines, each machine to put a portion of the data.
RabbitMQ is actually like, not distributed message queue, he is the message queue, but offers some clustering, HA mechanism of it, because no matter how play, RabbitMQ data is placed in a queue of a node Lane, mirroring the cluster, each node is put in the full data queue.
kafka 0.8 previously, there is no mechanism of HA is any broker is down, partition on that broker on the waste, can not write are unable to read, nothing to speak of high availability.
After kafka 0.8, it provides HA mechanism, the mechanism is the replica copy
- Each partition data will be synchronized to the other machine to form a plurality of replica copies of their own
- All the replica will then elect a leader out, then the production and consumption are related to dealing with this leader
- The other is the follower replica
In this way
- The time of writing, leader will be responsible for data synchronization up to all follower
- Read when reading the data directly to leader
Can read and write leader?
Very simple, if you are free to read and write each follower, then they would have to care about the issue of consistency, system complexity is too high, it is easy to go wrong
kafka will be distributed evenly to all replica of a partition on different machines, so it can improve fault tolerance.
So engaged, had called 高可用性up
because if a broker is down, all right, the broker above partition on the other machines have a copy if there is a partition of this top leader, then the time will be re-elected a new leader out, we continue to read and write the new leader can be. This so-called high availability
When writing data, the producer wrote leader, then leader will write data to a local disk floor, then other follower on their own initiative to pull data from the leader. Once all the data is good follower synchronization, it will be sent to the leader ack, ack after receipt of all follower of the leader, will return to write messages to the successful producer. (Of course, this is only one mode, this behavior may be appropriately adjusted)
Consumption of time, only from the leader to read, but only one message has been successfully synchronized all follower have returned ack when the news will be read by consumers.
In fact this mechanism, talk deep, can be very deep, but I still go back and locate the theme of our program, focused interview, at least you heard here generally understand how kafka is to ensure the high availability mechanism ,Right? We will not know anything about the site but also to the interviewer to draw chart. To encounter the interviewer is really kafka master, digging a question, then you can only say I'm sorry, you're not too deeply studied.
But we must understand that this thing is a trade-off, you are quick to spot common interview questions system, rather than in-depth study kafka, should thoroughly study kafka, you are not so much time. You can only be sure that, maybe you did not know that before this, but now you know, the interview was asked, you can probably talk about. Then a lot of other candidates, perhaps not as good as you, have not seen this, was asked simply did not answer, by contrast, point out that you can say, probably mean the
- Architecture as shown in FIG.
reference
"Java Engineer interview assault Season 1 - China huperzine teacher"
Please pay attention to dry more public JavaEdge No.