How to improve server concurrency

How to improve server concurrency, what is the theoretical basis and method

Hey, how much concurrency can a server handle? 10,000, 100,000, 110 million? What decides?

Needless to say, the final decision must be made by the hardware! Stronger cpu, larger memory, higher bandwidth, higher-speed io devices... With an IBM mainframe, even a rookie can reach a million concurrent

Please indicate the original work when reprinting: Lao Zhong in Blog Garden https://www.cnblogs.com/littlecarry/

It doesn't make much sense to discuss hardware

What we want to discuss is how to improve the concurrency capability of a single server under ordinary fixed hardware conditions.

Concurrency, that is, the maximum number of requests processed by the server per unit time, is usually defined as 1 second.

First of all, it is clear that improving the server's concurrency capability is a systemic problem that cannot be solved by one or two methods.

1. Return to the essence of the problem

In the face of external concurrent access, the behavior of the server is described as two points: connection request, processing request

1.1 Connection request

Thousands of requests are connected to the server at the same time, how to manage efficiently?

1.2 Processing requests

So many requests have to be processed, how to execute them quickly, and then return and release resources?

The shorter the execution time of a request, the more requests the server can process per unit time, and the stronger the concurrency capability.

2. What restricts processing speed

To improve the concurrency capability of the server, it is necessary to increase the processing speed of "connection requests" and "processing requests".

2.1 Improve the IO connection management capability of the operating system

Involved areas: blocking, non-blocking, synchronous, asynchronous IO, multiplexing, etc. When a request hits the server, the server needs to allocate resources such as memory, threads, and cpu. The sharing method can save a lot of resources.

Improvement method: Based on NIO multiplexing technology, hundreds of millions of requests can be connected to one server at the same time for efficient management

2.2 Improve the IO read and write capabilities of the operating system

Involved areas: network IO, hard disk IO, memory, etc. The reading and writing of io is very slow, which is a very important bottleneck. The execution of requests may involve network IO, hard disk IO, and memory reading and writing. Shortening or shielding these reading and writing will greatly increase the execution speed.

Improvement method: zero copy, DMA, cache, etc.

2.3 Improve system resource competition

Aspects involved: number of threads, lock competition, memory application release, system parameter configuration, etc.

Lifting method:

• 1) The more threads there are, the more the operating system consumes when switching thread contexts. If there are fewer threads, the blocking of IO may cause the system to idle. Allocate the right number of threads with actual debugging

• 2) There are many types of locks, such as pessimistic, optimistic, reentrant, etc. Choosing the right lock can increase the competition rate of resources. All locks are pessimistic locks, which are not advisable.

• 3) Reasonable application and release of memory can save resource consumption

• 4) System parameters, jvm parameters, many are allocated according to the actual situation

2.4 Improve the ability to write code

Involved areas: code structure, code logic, code operation on memory, etc.

Improvement method: write more codes, read more design patterns and architectures, and learn more about memory models

3. Summary:

The above four points can also be considered as performance bottlenecks

Many people on the Internet say that the concurrency of nginx is 20,000, 50,000, 60,000, etc. They are all wrong. One is that the code structure of nginx has no constraints on the concurrency, and the other is that it is separated from the hardware. The architecture of nginx does not restrict "connection requests" and "processing requests". It depends on your improvement of performance bottlenecks and your hardware. If these two points are improved, concurrent billions may not be a problem

Of course, there is also a "stand-alone performance" problem here. The operating system that does not need to switch threads will of course execute the fastest. Therefore, it is necessary to handle millions of concurrency, and the distributed processing efficiency is the highest. There are also disaster recovery, high availability and other issues

This article only provides basic theoretical ideas. Many specific methods will not be elaborated, so you have to inquire by yourself.

4. Related concepts

1. concept

• Number of concurrent connections - SBC (Simultaneous Browser Connections)

The number of concurrent connections means that the client initiates a request to the server and establishes a TCP connection. The total number of TCP connections to the server per second is the number of concurrent connections.

• Number of requests-QPS (Query Per Second)/RPS (Request Per Second)

There are two abbreviations for the number of requests, which can be called QPS or RPS. The unit is how many requests per second. Query=query, also equivalent to request. The number of requests refers to the fact that the client sends GET/POST/HEAD packets to the http service after the connection is established, and there are two situations after the server returns the request result:

1) The HTTP data packet header contains the word Close, which closes the current TCP connection;

2) The header of the http data packet contains the word Keep-Alive, the connection is not closed this time, and you can continue to send requests to the http service through this connection, which is used to reduce the number of concurrent TCP connections.

2. How to measure server performance?

Usually, what we test is QPS, which is the number of requests per second. However, in order to measure the overall performance of the server, it is best to test the number of concurrent connections and the number of requests together during the test.

• Test Principle

To test the number of concurrent connections, each concurrent 1 request is used, and multiple concurrent connections are performed;

The number of test requests is multi-concurrent and each concurrent multiple requests are carried out. The total number of requests will be = the number of concurrent requests * the number of single concurrent requests. It should be noted that the results obtained by different concurrent and single concurrent requests will be different, so It is best to test multiple times and take the average.

Reference:
https://www.51sjk.com/b19b329206/