How does the server handle thousands of user requests in parallel? What technologies are involved in this?
Single-server service model in high -concurrency scenarios
High-concurrency TCP long connections
multi-Progress
The earliest and simplest method in history to handle multiple requests concurrently is to use multiple processes.
For example, in the Linux world, we can use fork, exec and other methods to create multiple processes. We can receive the user's link request in the parent process, and then create a child process to handle the user request, just like this: 
the advantage of this method is Lies in:
- Programming is simple and very easy to understand
- Since the address space of each process is isolated from each other, a process crash will not affect other processes
- Make full use of multi-core resources
The advantages and disadvantages of multi-process parallel processing are obvious, but the disadvantages are also obvious:
- The address space of each process is isolated from each other, this advantage will also become a disadvantage, that is, it will become more difficult to communicate between processes, you need to use the interprocess communication (IPC, interprocesscommunications) mechanism, think about what you know now Inter-process communication mechanism, and then let you implement it in code? Obviously, inter-process communication programming is relatively complicated, and performance is also a major issue.
- The cost of creating a process is greater than that of a thread. Frequent creation and destruction of processes will undoubtedly increase the burden on the system.
Multithreading
Since threads share the process address space, communication between threads does not naturally require any communication mechanism, just read the memory directly.
The cost of thread creation and destruction is also reduced. You must know that threads are like hermit crabs. The house (address space) is a process, and it is only a tenant. Therefore, it is very lightweight and the cost of creation and destruction is also very small.
We can create a thread for each request, even if a thread is blocked due to performing I/O operations-such as reading the database, etc.-it will not affect other threads, just like this: 
but the thread is perfect, Does it cure all diseases? Obviously, the computer world has never been so simple.
Because the threads share the process address space, this brings convenience to the communication between threads and at the same time brings endless troubles.
It is precisely because the address space is shared between threads, a thread crash will cause the entire process to crash and exit. At the same time, the communication between threads is simply too simple, as simple as the communication between threads only needs to read the memory directly, and it is also simple enough to cause problems. It is extremely easy, such as deadlocks, synchronization and mutual exclusion between threads, etc. These are extremely prone to bugs, and a considerable part of the precious time of countless programmers is used to solve the endless problems caused by multithreading.
Although threads also have shortcomings, they are more advantageous than multi-processes. However, it is impractical to simply use multi-threading to solve high concurrency problems.
Because although the thread creation overhead is smaller than the process, it still has overhead. For a highly concurrent server with tens of thousands of hundreds of thousands of links, creating tens of thousands of threads will have performance problems, including memory usage and inter-threads. Handover, that is, the overhead of scheduling.
Therefore, we need to think further.
Linux, C/C++ technology exchange group: [960994558] I have compiled some learning books, interview questions from big companies, and technical teaching video materials that I think are better (including C/C++, Linux, Nginx, ZeroMQ, MySQL, Redis, fastdfs, MongoDB, ZK, streaming media, CDN, P2P, K8S, Docker, TCP/IP, coroutine, DPDK, etc.), you can add it yourself if you need it! ~
Event Loop: Event Driven
So far, when we mention the word "parallel", we think of processes and threads. However, parallel programming can only rely on these two technologies, it is not the case.
There is another parallel technology widely used in GUI programming and server programming. This is the very popular event-based concurrency in recent years.
Don't think this is a difficult technology to understand. In fact, the principle of event-driven programming is very simple.
This technique requires two raw materials:
- event
- A function that handles events, this function is usually called event handler
The rest is simple:
You just need to wait quietly for the event to arrive. When the event arrives, check the type of the event and find the corresponding event handler based on the type, and then call the event handler directly.
The above is the whole content of event-driven programming.
From the above discussion, we can see that we need to continuously receive events and then process events, so we need a loop (either while or for loops are fine), this loop is called Event loop.
This is how pseudocode is used:
while(true) {
event = getEvent();
handler(event);
}
What to do in the Event loop is actually very simple, just wait for the event to be brought, and then call the corresponding event processing function.
Note that this code only needs to run in one thread or process, and only this one event loop can handle multiple user requests at the same time.
Some students may still not understand why such an event loop can handle multiple requests at the same time?
The reason is simple. For a web server, most of the time when processing a user request is actually spent on I/O operations, such as database read and write, file read and write, network read and write, etc. When a request comes, it may need to query the database and other I/O operations after simple processing. We know that I/O is very slow. After initiating I/O, we can continue processing without waiting for the completion of the I/O operation. The next user request.
Now you should understand. Although the last user request has not been processed, we can actually process the next user request. This is parallelism. This kind of parallelism can be handled by event-driven programming.
This is like a waiter in a restaurant. A waiter cannot wait until the last customer places an order, serves food, eats, and pays before receiving the next customer. What does the waiter do? When a customer finishes placing an order, he will directly deal with the next customer, and when the customer has finished eating, he will come back to pay for the bill by himself.
As you can see, the same waiter can handle multiple customers at the same time. This waiter is equivalent to the Event loop here. Even if the event loop is only running in one thread (process), it can handle multiple user requests at the same time.
I believe you have a clear understanding of event-driven programming. Then the next question is event-driven and event-driven. Then how do you get this event, or event?
Event source: IO multiplexing
IO multiplexing technology is used to solve this problem. Through IO multiplexing technology, we can monitor multiple file descriptions at a time. When a file (socket) is readable or writable, we can Get notified.
In this way, the IO multiplexing technology has become the engine of the event loop, continuously providing us with various events, so that the source of the event is solved.
Problem: Blocking IO
Now, we can use one thread (process) to carry out parallel programming based on event-driven, and there are no more problems such as various locks, synchronization mutual exclusion, deadlocks, etc. that are annoying in multi-threading. However, there has never been a technology that can solve all problems in computer science. There is no technology now, and there will not be in the foreseeable future.
This IO method most commonly used by programmers is called blocking IO, that is to say, when we perform IO operations, such as reading files, if the file is not read, then our program (thread) will be blocked and Suspend execution, which is not a problem in multithreading, because the operating system can also schedule other threads.
But there is a problem in a single-threaded event loop. The reason is that when we perform blocking IO operations in the event loop, the entire thread (event loop) will be suspended. At this time, the operating system will have no other threads to call. Because there is only one event loop in the system processing user requests, so when the event loop thread is blocked and suspended, all user requests cannot be processed. You can imagine that your request is suspended when the server is processing other user requests to read the database. ?
Therefore, there is a precaution when event-driven programming is to not allow blocking IO to be initiated.
Some students may ask, if blocking IO cannot be initiated, then how to perform IO operations?
Where there is blocking IO, there is non-blocking IO.
Non-blocking IO
In order to overcome the problems caused by blocking IO, modern operating systems have begun to provide a new method of initiating IO requests. This method is asynchronous IO.
When asynchronous IO, suppose the aio_read function is called (for specific asynchronous IO API, please refer to the specific operating system platform), that is, asynchronous read. When we call this function, we can return immediately and continue other things, although the file may be It has not been read, so that it does not block the calling thread. In addition, the operating system also provides other methods for the calling thread to detect whether the IO operation is complete.
In this way, the IO blocking call problem is also solved with the help of the operating system.
Difficulties of event-based programming
Although there is asynchronous IO to solve the problem that the event loop may be blocked, event-based programming is still difficult.
First of all, we mentioned that the event loop runs in a thread. Obviously, a thread cannot make full use of multi-core resources. Some students may say that it is not enough to create multiple event loop instances. There are multiple event loop threads, but then the multithreading problem will appear again.
Asynchronous programming needs to combine callback functions. This programming method requires the processing logic to be divided into two parts, one part is handled by the caller itself, and the other part is handled in the callback function. This change in programming method increases the burden on programmers in understanding. Event-based programming will be difficult to expand and maintain later in the project.
So is there a better way?
To find a better way, we need to solve the essence of the problem, so what is the essence of the problem?
Better way
Why do we use asynchronous programming in such an incomprehensible way?
It is because blocking programming is easy to understand, but it causes the thread to be blocked and suspended.
Although event-based programming has various shortcomings, event-based programming is still very popular on today's high-performance and high-concurrency servers, but it is no longer purely event-driven based on a single thread, but event loop + multi thread + user level thread.
to sum up
High-concurrency technology has evolved from the initial multi-process to the current event-driven. Computer technology is also evolving and evolving just like biology, but in any case, understanding the history can give a deeper understanding of the present. I hope this article can help you understand high-concurrency servers.
