Article directory
1. Multiplexing IO
The term I/O multiplexing may be unfamiliar to some people, but it is easy to understand when it comes to select/epoll. In some scenarios, this I/O method is also called event-driven I/O (event-driven I/O). We all know that the advantage of select/epoll is that a single process can handle the I/O of multiple network connections at the same time. The basic principle is that the select/epoll function will continuously poll all the sockets in charge, and when a socket has data arriving, it will notify the user process. The flow is as follows:
When the user process calls select, the entire process will be blocked, and the kernel will monitor all the sockets that select is responsible for. When the data of any socket is ready, select will return. At this time, the user process calls the read operation again to copy the data from the kernel to the user process.
This process is actually not that different from blocking I/O, in fact it's slightly worse. Because two system calls (select and read) are needed here, and blocking I/O only calls one system call (read). But after using select, the biggest advantage is that users can process multiple socket I/O requests in one thread at the same time. Users can register multiple sockets, and then continuously call select to read the activated sockets, so as to process multiple I/O requests in the same thread at the same time. In the synchronous blocking model, this goal can only be achieved through multithreading. (By the way: So, if the number of connections processed is not very high, a web server using select/epoll is not necessarily better than a web server using multithreading + blocking I/O, and the delay may be greater. select The advantage of /epoll is not that it can process a single connection faster, but that it can process more connections.)
In the multiplexing model, for each socket, it is usually set to non-blocking (non-blocking), but, as shown above, the entire user process is actually blocked all the time. It's just that the process is blocked by the select function, not by socket I/O. Therefore, select() is similar to non-blocking I/O.
Most Unix/Linux operating systems support the select function, which is used to detect state changes of multiple file handles. The prototype of the select interface is given below:
FD_ZERO(int fd, fd_set* fds);
FD_SET(int fd, fd_set* fds);
FD_ISSET(int fd, fd_set* fds);
FD_CLR(int fd, fd_set* fds);
int select(int nfds, fd_set *readfds, fd_set *writefds, fd_set *exceptfds, struct timeval *timeout);
Here, the fd_set type can be simply understood as a queue of bitwise marked handles. For example, to mark a handle with a value of 16 in a certain fd_set, the 16th bit of the fd_set is marked as 1. The specific setting and verification can be realized by using FD_SET, FD_ISSET and other macros. In the select() function, readfds, writefds, and exceptfds are both input and output parameters. If the input readfds marks handle 16, select() will detect whether handle 16 is readable. After select () returns, you can check whether readfds marks handle No. 16 to determine whether the "readable" event has occurred. In addition, users can set the timeout time.
The following will re-simulate the model of receiving data from multiple clients in the above example.
The above model only describes the process of using the select() interface to receive data from multiple clients at the same time; since the select() interface can detect the read status, write status and error status of multiple handles at the same time, it can be easily constructed as A server system in which multiple clients provide independent question-and-answer services.
What needs to be pointed out here is that a connect() operation on the client side will trigger a "readable event" on the server side, so select() can also detect the connect() behavior from the client side.
In the above model, the most critical part is how to dynamically maintain the three parameters readfds, writefds and exceptfds of select(). As an input parameter, readfds should mark the handles of all "readable events" that need to be detected, which always include the "parent" handle that detects connect(); at the same time, writefds and exceptfds should mark all "writable events" that need to be detected and a handle to an "error event" (marked with FD_SET()).
As output parameters, readfds, writefds, and exceptfds store the handle values of all events captured by select(). Programmers need to check all flag bits (check with FD_ISSET()) to determine exactly which handles the event occurred on.
The above model mainly simulates the service process of "one question and one answer", so if select() finds that a certain handle has captured a "readable event", the server program should perform recv() operation in time and prepare according to the received data The data is to be sent, and the corresponding handle value is added to writefds to prepare for the next select() detection of the "writable event". Similarly, if select() finds that a certain handle has captured a "writable event", the program should perform the send() operation in time and be ready for the next "readable event" detection.
This model is characterized by detecting an event or set of events every execution cycle, and a specific event triggers a specific response. We can classify this model as an "event-driven model".
Compared with other models, the event-driven model using select() is only executed by a single thread (process), occupies less resources, does not consume too much CPU, and can provide services for multiple clients at the same time. If you try to build a simple event-driven server program, this model has certain reference value.
But this model still has many problems. First of all, the select() interface is not the best choice to implement "event-driven". Because when the value of the handle to be detected is large, the select() interface itself needs to consume a lot of time to poll each handle. Many operating systems provide more efficient interfaces, such as epoll provided by linux, kqueue provided by BSD, and /dev/poll provided by Solaris. If you need to implement a more efficient server program, an interface like epoll is more recommended. Unfortunately, the epoll interfaces specially provided by different operating systems are very different, so it will be difficult to use an interface similar to epoll to implement a server with better cross-platform capabilities.
Second, this model mixes event detection and event response. Once the execution body of event response is huge, it will be catastrophic to the entire model. In the following example, the huge execution body 1 will directly cause the delay in execution of the execution body responding to event 2, and greatly reduce the timeliness of event detection.
Fortunately, there are many efficient event-driven libraries that can shield the above-mentioned difficulties. Common event-driven libraries include libevent library and libev library as a libevent replacement. These libraries will select the most appropriate event detection interface according to the characteristics of the operating system, and add technologies such as signals to support asynchronous responses, which makes these libraries the best choice for building event-driven models. The next chapter will introduce how to use the libev library to replace the select or epoll
interface to achieve an efficient and stable server model.
In fact, starting from 2.6, the Linux kernel also introduces IO operations that support asynchronous responses, such as aio_read, aio_write, which is asynchronous IO. The advantage of asynchronous IO is that the application does not need to wait for the completion of the IO operation, but is notified after the IO operation is completed. This allows the application to process other tasks while waiting for an IO operation to complete, thereby improving the execution efficiency of the application.
In short, by using IO multiplexing technologies such as select, epoll, kqueue or asynchronous IO, event-driven server programs can be implemented to improve the performance and scalability of server programs. In practical applications, you can choose an appropriate event-driven library or asynchronous IO method according to specific needs and platform characteristics to meet the needs of different scenarios.
There are a few other key factors to consider when building a high-performance, scalable server program. Here are some suggestions and tips:
- Load balancing: When faced with a large number of client connections, it is very important to distribute the load reasonably. Load balancing algorithms such as round robin, least connections, and consistent hashing can be used to distribute client requests to different server nodes to achieve load balancing.
- Caching: Caching is a key means of improving server performance. Caching frequently accessed data or calculation results can reduce the computing load and response time of the server. Technologies such as local cache and distributed cache can be used, such as Memcached, Redis, etc.
- Connection pool: In order to reduce the overhead of establishing and disconnecting connections, you can use connection pooling technology. A connection pool pre-establishes a certain number of connections and distributes connections to requests as needed. This method can reduce the time overhead of connection establishment and improve server performance.
- Thread and process management: Reasonable management of threads and processes in the server program can improve the processing capacity of the server. Technologies such as thread pools and process pools can be used to avoid frequently creating and destroying threads or processes.
- Optimizing data structures and algorithms: Optimizing data structures and algorithms is the basis for improving server program performance. Using appropriate data structures and algorithms can greatly improve the execution efficiency of programs.
- Distributed and microservice architecture: With the development of business and the increase of system complexity, distributed architecture and microservice architecture can be considered. These architectures split the system into multiple independent modules, and each module is responsible for a specific function, thereby improving the scalability and maintainability of the entire system.
- Performance monitoring and tuning: regularly monitor the performance of server programs, analyze performance bottlenecks, and perform corresponding tuning. You can use performance monitoring tools, log analysis and other methods to find out performance problems in the system and optimize them.
Through the above methods and techniques, high-performance, scalable event-driven server programs can be built in different scenarios. In practical applications, appropriate technologies and strategies are selected according to specific needs and scenarios to meet the performance and scalability requirements of server programs.
Example of creating a simple TCP echo server using epoll:
It can handle multiple client connections at the same time, the server uses to epoll_create1create an epoll instance, and then uses epoll_ctlto add the listening socket and client socket to the epoll instance. Use epoll_waitto wait for file descriptor state changes, and then handle those events.
epoll.cpp
#include <iostream>
#include <sys/socket.h>
#include <netinet/in.h>
#include <unistd.h>
#include <fcntl.h>
#include <sys/epoll.h>
#include <string.h>
const int MAX_EVENTS = 10;
int make_socket_non_blocking(int sfd) {
int flags = fcntl(sfd, F_GETFL, 0);
if (flags == -1) {
std::cerr << "Error: fcntl failed" << std::endl;
return -1;
}
flags |= O_NONBLOCK;
if (fcntl(sfd, F_SETFL, flags) == -1) {
std::cerr << "Error: fcntl failed" << std::endl;
return -1;
}
return 0;
}
int main() {
int sfd = socket(AF_INET, SOCK_STREAM, 0);
if (sfd == -1) {
std::cerr << "Error: socket creation failed" << std::endl;
return 1;
}
if (make_socket_non_blocking(sfd) == -1) {
close(sfd);
return 1;
}
sockaddr_in addr;
memset(&addr, 0, sizeof(addr));
addr.sin_family = AF_INET;
addr.sin_addr.s_addr = INADDR_ANY;
addr.sin_port = htons(8080);
if (bind(sfd, (sockaddr*)&addr, sizeof(addr)) == -1) {
std::cerr << "Error: bind failed" << std::endl;
close(sfd);
return 1;
}
if (listen(sfd, SOMAXCONN) == -1) {
std::cerr << "Error: listen failed" << std::endl;
close(sfd);
return 1;
}
int efd = epoll_create1(0);
if (efd == -1) {
std::cerr << "Error: epoll_create1 failed" << std::endl;
close(sfd);
return 1;
}
epoll_event event;
event.data.fd = sfd;
event.events = EPOLLIN | EPOLLET;
if (epoll_ctl(efd, EPOLL_CTL_ADD, sfd, &event) == -1) {
std::cerr << "Error: epoll_ctl failed" << std::endl;
close(sfd);
close(efd);
return 1;
}
epoll_event events[MAX_EVENTS];
while (true) {
int n = epoll_wait(efd, events, MAX_EVENTS, -1);
for (int i = 0; i < n; ++i) {
if (events[i].data.fd == sfd) {
// 新连接
while (true) {
sockaddr in_addr;
socklen_t in_len = sizeof(in_addr);
int infd = accept(sfd, &in_addr, &in_len);
if (infd == -1) {
break;
}
if (make_socket_non_blocking(infd) == -1) {
close(infd);
continue;
}
epoll_event event;
event.data.fd = infd;
event.events = EPOLLIN | EPOLLET;
if (epoll_ctl(efd, EPOLL_CTL_ADD, infd, &event) == -1) {
std::cerr << "Error: epoll_ctl failed" << std::endl;
close(infd);
continue;
}
}
} else {
// 处理已连接的客户端
char buf[1024];
ssize_t count = 0;
while (true) {
count = read(events[i].data.fd, buf, sizeof(buf));
if (count <= 0) {
break;
}
// 回显数据
ssize_t written = 0;
while (written < count) {
ssize_t n = write(events[i].data.fd, buf + written, count - written);
if (n == -1) {
break;
}
written += n;
}
}
if (count == 0 || (count == -1 && errno != EAGAIN)) {
// 断开连接
close(events[i].data.fd);
epoll_ctl(efd, EPOLL_CTL_DEL, events[i].data.fd, nullptr);
}
}
}
}
close(sfd);
close(efd);
return 0;
}
client.cc for testing:
#include <iostream>
#include <sys/socket.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#include <unistd.h>
#include <string.h>
int main() {
int sockfd = socket(AF_INET, SOCK_STREAM, 0);
if (sockfd == -1) {
std::cerr << "Error: socket creation failed" << std::endl;
return 1;
}
sockaddr_in addr;
memset(&addr, 0, sizeof(addr));
addr.sin_family = AF_INET;
addr.sin_port = htons(8080);
if (inet_pton(AF_INET, "127.0.0.1", &addr.sin_addr) <= 0) {
std::cerr << "Error: inet_pton failed" << std::endl;
close(sockfd);
return 1;
}
if (connect(sockfd, (sockaddr*)&addr, sizeof(addr)) == -1) {
std::cerr << "Error: connect failed" << std::endl;
close(sockfd);
return 1;
}
std::string message;
char buffer[1024];
while (true) {
std::cout << "Enter message to send: ";
std::getline(std::cin, message);
if (message == "exit") {
break;
}
ssize_t sent = send(sockfd, message.c_str(), message.size(), 0);
if (sent == -1) {
std::cerr << "Error: send failed" << std::endl;
break;
}
ssize_t received = recv(sockfd, buffer, sizeof(buffer), 0);
if (received == -1) {
std::cerr << "Error: recv failed" << std::endl;
break;
}
std::cout << "Server response: ";
std::cout.write(buffer, received);
std::cout << std::endl;
}
close(sockfd);
return 0;
}
2. Asynchronous I/O
Under Linux, asynchronous IO (asynchronous IO) is mainly used for disk IO read and write operations, not network IO. Introduced starting from version 2.6 of the kernel. First look at the process of asynchronous IO:
After the user process initiates a read (read) operation, it can immediately begin to perform other tasks. From the perspective of the kernel (kernel), when it receives an asynchronous read request (asynchronous read), it will return immediately, so it will not cause any blocking (block) to the user process. Next, the kernel waits for the data preparation to complete before copying the data to user memory. When all these operations are completed, the kernel will send a signal (signal) to the user process to notify it that the read operation has been completed.
As for servers implemented using asynchronous IO, no examples are given here. Later, when there is time, I can open another article about it. Asynchronous IO is truly non-blocking, it will not block the request process, so it is very important for the implementation of high-concurrency web servers.
So far, four IO models have been introduced. Now back to the first few questions: What is the difference between blocking and non-blocking? What is the difference between synchronous IO (synchronous IO) and asynchronous IO (asynchronous IO)?
Answer the simplest question first: blocking vs. non-blocking. The difference between the two has been clearly explained in the previous introduction. Calling blocking IO will block the corresponding process until the operation is completed; non-blocking IO will return immediately when the kernel is still preparing data.
The difference between the two is that synchronous IO will block the process when performing "IO operations". According to this definition, the aforementioned blocking IO, non-blocking IO, and IO multiplexing (IO multiplexing) all belong to synchronous IO. One could argue that non-blocking IO is not blocked. There is a very "cunning" place here. The "IO operation" referred to in the definition refers to the real IO operation, that is, the read (read) system call in the example. When non-blocking IO executes the read system call, if the kernel data is not ready, the process will not be blocked at this time. But when the data in the kernel is ready, the read operation will copy the data from the kernel to the user memory, and the process is blocked at this time. The asynchronous IO is different. When the process initiates the IO operation, it returns directly and ignores it until the kernel sends a signal to tell the process that the IO has been completed. During this whole process, the process is not blocked at all.
asioExample of creating an asynchronous TCP echo server using the library in C++ :
-
Install
asiothe library:-
Install via package manager:
For some Linux distributions,asiothe library may already be included in the software repositories. For example, on Debian-based systems such as Ubuntu, it can beaptinstalled using the package managerlibasio-dev:sudo apt-get update sudo apt-get install libasio-devOn Fedora-based systems, it can be
dnfinstalled using the package managerasio-devel:sudo dnf install asio-develLibraries installed through the package manager
asioare automatically added to the system's header file search path, so there is no need to specify them manually. -
Manual installation:
If your Linux distribution does not have pre-packagedasiolibraries, or you wish to install a specific version ofasiothe library, you can download and install it manually.First, visit
asiothe official website of the library: https://think-async.com/Asio/, click "Standalone" in the "Download" section to download the standalone version of the asio library. This version does not depend on the Boost library, but requires a compiler with C++11 support.After the download is complete, unzip the file.
Next,
asiocopy the entire folder into your project directory, or addasio/includethe directory to your compiler/build system's header file search path.Taking the g++ compiler as an example, it will
asio/includebe added to the header file search path, and-Ithe header file path can be specified with the option:g++ -std=c++11 -o server server.cc -I/home/ricky/asio/asio-1.26.0/include -lpthreadSupplement:
-IThe option specifies the header file path,-Lthe option specifies the library file path, and-lthe option specifies the library file to be linked
-
-
Asynchronous TCP echo server code:
#include <iostream> #include <asio.hpp> #include <memory> #include <thread> #include <chrono> using asio::ip::tcp; class EchoSession : public std::enable_shared_from_this<EchoSession> { public: EchoSession(tcp::socket socket) : socket_(std::move(socket)) { } void start() { read(); } private: void read() { auto self(shared_from_this()); socket_.async_read_some( asio::buffer(data_, max_length), [this, self](const std::error_code& error, std::size_t bytes_transferred) { if (!error) { write(data_, bytes_transferred); } }); } void write(const char* data, std::size_t bytes_transferred) { auto self(shared_from_this()); asio::async_write( socket_, asio::buffer(data, bytes_transferred), [this, self](const std::error_code& error, std::size_t /*bytes_transferred*/) { if (!error) { read(); } }); } tcp::socket socket_; enum { max_length = 1024 }; char data_[max_length]; }; class EchoServer { public: EchoServer(asio::io_context& io_context, short port) : acceptor_(io_context, tcp::endpoint(tcp::v4(), port)) { accept(); } private: void accept() { acceptor_.async_accept( [this](const std::error_code& error, tcp::socket socket) { if (!error) { std::make_shared<EchoSession>(std::move(socket))->start(); } accept(); }); } tcp::acceptor acceptor_; }; int main(int argc, char* argv[]) { try { if (argc != 2) { std::cerr << "Usage: async_tcp_echo_server <port>\n"; return 1; } asio::io_context io_context; EchoServer server(io_context, std::atoi(argv[1])); io_context.run(); } catch (std::exception& e) { std::cerr << "Exception: " << e.what() << "\n"; } return 0; } -
Compile the code:
g++ -std=c++11 -o server server.cc -I/home/ricky/asio/asio-1.26.0/include -lpthread -
Run server:
Run the compiled executable and specify a port number. For example, using port numbers8080:./server 8080 -
test server
telnet localhost 8080
3. Signal driven IO
First, we need to allow the socket to perform Signal-Driven I/O (Signal-Driven I/O) and install a signal handler. This way, the process can continue running without being blocked. When data is ready, the process receives a SIGIO signal. In the signal processing function, we can call the I/O operation function to process the data. When a datagram is ready to be read, the kernel generates a SIGIO signal for the process. We can call the read function in the signal processing function to read the datagram, and notify the main loop that the data is ready to be processed; we can also immediately notify the main loop to let it read the datagram. No matter how the SIGIO signal is handled, the advantage of this model is that the process can continue to execute without being blocked while waiting for the datagram to arrive (the first phase). This avoids the blocking and polling of select, and when there is an active socket, it is processed by the registered handler.
Through the above introduction, we can find that the difference between non-blocking IO (non-blocking IO) and asynchronous IO (asynchronous IO) is obvious. In non-blocking IO, although the process will not be blocked most of the time, it still needs to actively check, and after the data preparation is completed, the process also needs to actively call the recvfrom function again to copy the data to the user memory. Asynchronous IO is completely different. It is like the user process handing over the entire IO operation to others (the kernel) to complete, and then after the operation is completed, the kernel notifies the user process through a signal. During this period, the user process does not need to check the status of the IO operation, nor does it need to actively copy data.
An example of using signal-driven I/O to handle TCP connections in C++:
Signal-driven I/O depends on the signal mechanism of the operating system. In C++, the <signal.h> library can be used to implement signal-driven I/O.
#include <iostream>
#include <signal.h>
#include <fcntl.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <unistd.h>
#include <cstring>
void sigio_handler(int sig);
int sockfd;
int main() {
struct sockaddr_in server_addr;
sockfd = socket(AF_INET, SOCK_STREAM, 0);
if (sockfd < 0) {
std::cerr << "Error: Unable to create socket" << std::endl;
return 1;
}
server_addr.sin_family = AF_INET;
server_addr.sin_addr.s_addr = INADDR_ANY;
server_addr.sin_port = htons(8080);
if (bind(sockfd, (struct sockaddr *)&server_addr, sizeof(server_addr)) < 0) {
std::cerr << "Error: Unable to bind socket" << std::endl;
return 1;
}
if (listen(sockfd, 5) < 0) {
std::cerr << "Error: Unable to listen on socket" << std::endl;
return 1;
}
signal(SIGIO, sigio_handler);
fcntl(sockfd, F_SETOWN, getpid());
int flags = fcntl(sockfd, F_GETFL);
fcntl(sockfd, F_SETFL, flags | O_ASYNC);
while (1) {
pause();
}
return 0;
}
void sigio_handler(int sig) {
struct sockaddr_in client_addr;
socklen_t client_addr_len = sizeof(client_addr);
int client_socket = accept(sockfd, (struct sockaddr *)&client_addr, &client_addr_len);
if (client_socket < 0) {
std::cerr << "Error: Unable to accept connection" << std::endl;
return;
}
char buffer[256];
memset(buffer, 0, 256);
int n = read(client_socket, buffer, 255);
if (n < 0) {
std::cerr << "Error: Unable to read from client" << std::endl;
return;
}
std::cout << "Received message: " << buffer << std::endl;
close(client_socket);
}
To test this TCP server, this time we use telnet or the nc command to connect to the server from the command line:
Open a new command line terminal and run the following command:
telnet localhost 8080
or
nc localhost 8080
The command line will try to connect to the server. Once connected, you can enter some text and press enter to send it to the server. The server will output received messages to the console.
server:
