Architecture design Step 2: Design alternatives
The work of architect is not a mystery, mature architect need to be very familiar with the technology already exists, and already proven architectural patterns by heart, then according to their own understanding of the business , choosing the right architecture model combined , again the combined scheme modifications and adaptations .
For example, the primary and high-availability solutions, clustering solutions, high-performance load balancing, multiplexing, layered scalable, plug-in technologies such as after, most of the time we have clear goals, what you want to be able to find alternative solutions.
Only when this way completely unable to meet the demand, will consider innovative solutions, and in fact the vast majority of innovation under the program are also based on existing proven technology.
NoSQL : storage of Key-Value and index database is actually similar to the, Memcache just a database of indexed independent out to make a caching system .
Hadoop large file storage solutions , in fact, is the basis of the cluster program + data replication scheme.
Docker virtualization, the foundation is LXC (Linux Containers).
LevelDB file storage structure is the Skip List.
The new technology is in the prior art developed on the basis of prior art has come from previous techniques. The technique of functional groups, can greatly simplify the design process, this technology is "modular" in the first place. Technology "combined" and "recursive" feature that will revolutionize our understanding of the nature of technology.
Although the combination based on existing technology or infrastructure mode, and then adjusted, in most cases it is possible to get the program we need, but it does not mean that architecture is a very simple matter. Because the choices are many, the combination of programs more often the solution of a problem there are many; if we make some innovation in the combination of programs, solutions will be more. Therefore, how to design the final solution is not an easy thing, this stage is a lot of architects fallible place.
The first common mistake: design the best solution.
Many architects in the design of the architecture program, and my heart there is a technical default complex: I want to design a good architecture, in order to reflect my technical ability! For example, high availability scenario, clustering scheme clearly than the standby program to be excellent and powerful; a high-performance solution, Taobao XX program is the industry's leading solutions ...... architecture design principles "right principles" and "based on the simple principle of "requirements
The second common mistake: only a program.
Many architects do design, you may simply be the heart of several programs initial idea, then easily determine which is best, and then based on this judgment to begin detailed architectural design.
This has many drawbacks :
My heart is too simple assessment, may not want to have a comprehensive, just because one drawback scheme to put a veto, while in fact no program is perfect, somewhere flawed approach may be the best the whole program .
Architect again how cattle, empirical knowledge and skills have their limits, there may be a standard or empirical evaluation is incorrect, or old experience is not suitable for the new situation, even some evaluation criteria is to understand their original architect wrong.
Single design will appear excessive defense case, that is when the architecture review program for existing problems and questions, architects will make every effort to defend their own design, inexperienced designer might be irrational.
The number of alternatives to the best three to five . Less than three programs may be due to narrow thinking, poorly thought; more than five you need to spend a lot of time and energy, and the difference between the program may not be obvious.
Differences options to be more obvious . For example, standby program and program cluster difference is very apparent, or a standby program likewise, with ZooKeeper shots making and using standby Keepalived shots differential backup decision is obvious. But decisions are prepared with ZooKeeper call the shots, a detection period is one minute, a detection period is five minutes, and this difference is not architecture, but differences in the details, is not suitable to make the two programs.
Technical options have been limited not only familiar with the technology . When designing architecture, architects need vision to relax , consider more possibilities. Many architects or designers accumulated some successful experience, for the purpose of quickly complete tasks and reduce risk, may consciously or unconsciously, tend to use their own already familiar with the technology, the new technology there is a feeling not at ease. Like the old adage says: " If your hands are a hammer, every problem is a nail in your opinion ." For example, architects are familiar with MySQL, so no matter what storage solutions are based on MySQL to design, system performance is not enough , the first consideration is MySQL sub-library sub-table , and in fact perhaps the introduction of a Memcache cache will be able to solve the problem.
A third common mistake: too detailed options.
To focus attention on the details, ignoring the overall technical design, resulting in an insufficient number of alternatives or little difference.
Reviewed when other people will be a lot of details to go around, poor assessment results. For example, when a timer should be reviewed for 1 minute or 30 seconds, debate ruthlessly.
The correct approach is concerned that the alternative stage technology selection , rather than the technical details, differences in technology selection should be more obvious.
The actual design alternatives
Or return to the "forward waves micro-Bo," the scene, the last issue we through the "investigation method" recognition of the complexity of the message queue is mainly reflected in: high-performance message read take, high availability message written in, high availability messaging deposit storage, high availability messaging reading take. Subsequently to Step 2, design alternatives.
1. Option 1: Open source Kafka
Kafka is a mature open source program message queue, powerful, performance is very high, and relatively mature, many large companies are using.
2. Option 2: The cluster storage + MySQL
First, consider a single server high performance . Reading the message belongs to the category of high-performance "high availability computing", there are a variety of high-performance alternative to a single server. Considering the team's development language is Java, although some people think that C / C ++ language more appropriate to write high-performance middleware system, but architects together, consider it necessary to the performance advantages of language and let the whole team to switch languages, message queuing systems continue to use Java development. Since Netty mature Java art high-performance network database , so architects developing Netty selected based message queuing system.
Because the system design QPS is 13800 , even though a stand-alone system using Netty to build high-performance, single server to support such a high QPS still very risky, so the architects choose to take the cluster approach to meet the high performance message read , cluster load balancing algorithm uses a simple poll can .
Similarly, "high availability write" and "high performance read," Like , can take the form of clusters to meet. Because as long as the message is written to one of the servers in the cluster even if successful write, so "highly available write" allocation algorithm clusters and "high performance read" the same polling that under normal circumstances, the client will in turn write the message the different servers; abnormality under a server, the client writes the message directly to the next stage of the normal server.
The entire system most complicated is the "availability store " and "availability read " message "memory availability" requirements have been written in the case where a single server downtime is not lost ; "read availability" requires message may have been written in the case where a single server downtime continues to read . Architects first thought that may utilize the MySQL standby replication to achieve "highly available storage" by standby server program to achieve "high availability read" purposes.
specific plan:
A brief description of the program:
Using dispersed clustered data schema, server cluster packets, each data portion of the message store.
Each packet contains a primary and a backup MySQL MySQL, standby copy data in the packet, the packet data is not synchronized between .
Normally, the master server within the packet and provide external write message service message reader, the server is not prepared to provide services; if the primary server goes down, the backup server to provide services the read message.
Client adopt written policies poll and read messages.
3. Option 3: + cluster storage solution from Advantech
On the basis of option 2, based on the MySQL storage Replace self-development to achieve storage scheme, because the characteristics of the MySQL relational database is not very fit the characteristics of the message queue data, reference Kafka practice, can achieve their own set of file storage and copy program (specific embodiment described herein will be omitted, the program needs to be given the actual design).
As can be seen, a plurality of high-performance message is not read when the optional single alternative design of this part of the system, Option 2 and Option 3 are taken based Netty network library , developed using the Java language, the reason is that team of Java background constraint alternative range. Typically, mature team will not easily change technology stack, but the newly established technical team is more inclined to adopt new technologies.
The above simply gives three options for exemplary how, in practice, complicated than the above scheme. The richer the architect of the technical reserves, more experience, more options will be, so they can better design alternatives. For example, open-source program may choose to include Kafka used to live, ActiveMQ, RabbitMQ ; clustered storage solutions can only consider using MySQL , you can also consider using HBase , you can also consider using Redis and MySQL combination etc; since the inquiry file system can have multiple, can refer to Kafka, also read LevelDB, and the like can also refer to HBase. Due to space limitations, not one by one started.
summary
In addition to these three options, if allow you to design a fourth alternative, what is your plan?
Comments 1
In addition, the text option. 1: RabbitMQ than Kafka more mature, more reliable ; high performance indicators, Kafka winning, Kafka originally designed to process logs, more suitable for high-throughput processing of IO . However, "before the wave microblogging" system QPS requirements, RabbitMQ can also control . Tend RabiitMQ.
2 Comments
Message storage and high availability can be read using mongo and redis so high gps is very difficult to ensure that messages are not lost you can have a retroactive MQ messaging and message confirmation mechanism using or from time to time to consider research rpc message failed node and then reselect place an order
3 Comments
Use Kafka, Rabbitmq will Netty + Mysql than self-study much better. The scene of the real-time requirement is still relatively high (in general message queue active push mode ). Open source message queue has on consumers Push Profile ( Kafka used to live is also the time of consumption take pull mode instead of push mode ) . Self-development , then, if the message push, the Message Queuing service to all consumers need to record status information on the server side, but also consider the various exceptions and message confirmation, should be to implement very complex . The company generally did not have the technical strength.
Comments 4?
Option II architecture diagram can be understood as the sub-library standby program , or do all databases are full amount , I understand the former, because the text says no synchronization between groups , then missing here a according to message routing module , right
Author: hedgehog1112
link: https: //www.jianshu.com/p/46002d6b0a88
Source: Jane books
are copyrighted by the author. Commercial reprint please contact the author authorized, non-commercial reprint please indicate the source.