Distributed Architect: These are the things I need to know

Some time ago, I saw an article on the Internet about what kind of capabilities you need to have as an architect, but it includes the following points:

1. For the Java basic technology system (including JVM, class loading mechanism, multiple Thread concurrency, IO, network) have certain mastery and application experience.

2. Have a clear understanding of object-oriented software development ideas, and be familiar with common design patterns;

3. Be proficient in the current popular open source frameworks (spring/springmvc/ibatis), and have a certain understanding of its core ideas and implementation principles;

4. Familiar with the development and design of databases such as Oracle and MySQL, as well as the design and development of cache systems REDIS or Memcached;

5. Familiar with underlying middleware and distributed technologies (including cache, message system, hot deployment, JMX, etc.)

6. At least one Java application server such as tomcat

7, proficient in shell programming, proficient in the application of awk, sed, grep, strace, tcudump, gdb and other common commands;

8, with large-scale distributed, high concurrency, high load (large data volume), high availability system design Development experience

9. Have some understanding of configuration management and agile R&D model

10. Business capabilities

Of course, the above ten points are all capabilities that an advanced architecture should master or already possess, even if we know that architects need such capabilities Well, in fact, it is still far from the stage of architect, just like when you were in the college entrance examination, you knew that you had to take mathematics, physics, chemistry, politics, history and geography.

We know that these are the subjects of the exam, and how to get these things or how to deal with each knowledge point in the college entrance examination, so after knowing the direction of technology, of course, we should learn the specific knowledge inside; Structure this topic to introduce what specific knowledge points are needed, and how do you obtain these knowledge points;

1. Communication

Since it is a distributed system, it is inevitable to master the technology of communication between systems.

First of all, you must master some basic knowledge, such as network communication protocols (such as TCP/UDP, etc.), network IO (Blocking-IO, NonBlocking-IO, Asyn-IO), network cards (multi-queue, etc.); more application level, need Learn for example connection multiplexing, serialization/deserialization, RPC, load balancing, etc.

Communication connection method:

There are usually two ways for a large number of connections:

1. A large number of clients are connected to a server.

Now that NonBlocking-IO is so mature, it is not so difficult to write a server that supports a large number of clients.

There is a point to pay special attention to, that is, when the server hangs, it cannot appear that all clients initiate reconnection at one point in time, which is basically a disaster.

Usually, the method that can be used is that the client sleeps for a random time before reconnecting, and the other is to adopt an avoidance algorithm for the reconnection interval.

2. Scalability

Distributed systems basically mean that the scale is not small. For such systems, scalability issues must be considered when designing such systems. For any point drawn on the architecture diagram, if the amount of requests or data continues to increase, How to do it can be solved by adding machines. Of course, this process does not need to consider infinite scenarios. If you have experienced architects ranging from small to very large, obviously the advantage is not small, and it will also be more and more important. increasingly scarce.

The problem of scalability is solved around the following two scenarios:

1. Stateless scenarios

Stateless scenarios usually put a lot of state in the db, and after a certain stage, it will be necessary to introduce service to alleviate the excessive number of db connections. condition.

2. Stateful Scenarios

The so-called state is actually data. Sharding is usually used to achieve scalability. Sharding has a variety of implementation methods. The common ones are as follows:

2.1 Rule Sharding

2.2 Consistent Hash

2.3Auto Sharding It is OK to add machines as the volume increases, but usually in the case of Auto Sharding, there will be relatively high requirements on how to use it,

and this will usually cause some restrictions, such as HBase.

2.4CopyCopy is common in the situation where there are far more reads than writes. When implemented, there will be an eventually consistent solution and a globally consistent solution. Most of the eventually consistent ones can be passed through the message mechanism, etc.

The globally consistent ones such as zookeeper/etcd, etc. It is difficult to achieve both global consistency and high write support capability.

3. Stability

As a distributed system, it is necessary to consider how to deal with any point in the whole system if it hangs up (when a certain machine scale is reached, it is normal to hang up some machines every day). It is also mainly divided into stateless and stateful:

1. Stateless scenarios For stateless scenarios, it is usually easy to handle. It is OK to use only the detection mechanism such as heartbeat on the node discovery mechanism. From experience, it is nothing more than that relying solely on 4-layer detection is not enough for the business, usually it has to be done 7-layer, of course, to make 7-layer, you have to deal with the problem of large

scale Which machine becomes the master, such as the implementation based on paxos. Maintainability How the entire system environment should be built, deployed, supporting maintenance tools, monitoring points, alarm points, problem location, problem handling strategies, etc.

Fourth, maintainability

Maintainability is a part that is easily missed, but it is actually a very important part for distributed systems, such as how the entire system environment should be built, deployed, supporting maintenance tools, monitoring points, alarm points, problem location, and problem handling strategy, etc.

Fifth, of course, the most important thing is the following knowledge, you want to become an architect but don't know how to start 5-6-1-6-1-4-3-0-5 This is a buckle group for architecture Knowledge I will update a knowledge point for free in the group every day. If you think these knowledge points can help you, I look forward to your arrival.

Of course, the more detailed knowledge points are in the following topic
[img][/img]

These are only the knowledge that needs to be mastered for distributed topics, and we have six major sections of knowledge topics in the entire architect's topic. All dry stuff.