Architecture: Resource isolation.

What is resource isolation?

Service isolation means that in a large-scale system, the components, modules, services, and resources that are connected together can be separated. Then when the system fails, the fault can be isolated to prevent the propagation, and there will be no snowball and avalanche effects.

The main ways of isolation are

• Thread isolation

• Process isolation

• Cluster isolation

• Computer room isolation

• Read and write isolation

• Dynamic and static isolation

• Reptile isolation

• Resource isolation

• Wait....

Thread isolation

Mainly in a multithreaded environment, the thread pool is managed to separate the core business from the non-core business.

For example

 

Resource isolation in Tomcat

 

problem

1. Once a resource problem occurs, although it is in an isolated state, it is difficult to make the resource available again without restarting the jvm.

2. If the internal threads of the thread pool have OOM, FullGC, cpu exhaustion and other problems, it is also uncontrollable

in conclusion

Thread isolation can only guarantee isolation on the resource of the allocated thread, but cannot guarantee the overall stability

Process isolation

In the initial stage of the project, it is generally allinone's technical architecture, and then load balancing and session sharing are performed.

Avalanches cannot be completely avoided through thread isolation.

Resources such as java cpu and memory can be isolated by different virtual machine processes.

1. Cluster

2. distributed

Cluster isolation

If a business module in the system contains

• Panic buying, spike

• High storage I/O intensive

• High network I/O

• Calculate the high I/O

When this type of demand is required, it is easy to use this function to exhaust all the resources occupied by the entire module when the amount of concurrency is high, resulting in response coding or even node unavailability.

solution

• Independent split module

• Microservices

You can use hystrix to isolate distributed service failures in microservices. He can be isolated by threads and semaphores.

Isolate the computer room and live in different places

Solve the problems of large data capacity, high computing and I/O (network) density

Establish an overall copy of the service (computing services, data storage), and do multiple live or cold backups in multiple computer rooms, which is an enlarged version of the heterogeneous microservice data

When there is a problem at the computer room level, you can quickly switch through smart dns, httpdns, load balancing and other technologies

Data (read and write) separation

Through the master-slave mode, data storage services such as mysql and redis are clustered, and read and write are separated. Then, when the written data is not available, the data can be temporarily read through other nodes through the retry mechanism .

When multi-nodes are doing subnetting, in addition to multiple activities in different places, they can also be used as data centers. All data in the local computer room crud asynchronously and synchronously to the data center, and the data center distributes the data to other computer rooms.

Then when the data is temporarily unavailable in the local computer room, you can try to connect to the remote computer room or data center.

Dynamic and static isolation

Isolate static resources from original services, refer to cdn technology

Reptile isolation

Currently we are developing API interfaces, and most of them are open API interfaces. In other words, as long as someone gets this interface, anyone can get data through this API interface. Like a web crawler, the request speed is fast and the data obtained is too much. It will not take long for the crawler to use our API. Interface to develop a same website, in this case, the consequences are a bit serious, so we need to limit the flow and limit the frequency of access

The frequency of API interface calls of the open platform needs to be limited to save server resources and avoid malicious frequent calls

In large-scale Internet projects, the access flow for web services and web crawlers can reach 5:1, or even higher. Sometimes, some systems may run out of resources and services are unavailable due to high crawler traffic.

Current limiting dimension

• Login/session restrictions

• Download limit

• Visit frequency

• ip restriction, black and white list

To tell if a visit is a crawler, you can simply use nginx to analyze the lua processing

Nginx can not only process lua to separate traffic, but also complete more complex logic through the more powerful openresty, and realize a traffic gateway and soft firewall.

Resource isolation

• Disk

• database