foreword
The trigger mode of epoll is a topic that has caused a lot of discussion. There are also many articles summarized on the Internet. First of all, the name is not very uniform. LT mode is often called level trigger, level trigger, and conditional trigger, while ET mode It is often referred to as edge-triggered, edge-triggered, etc. These are translated from English, but there are some differences in translation. The full name of LT is level-triggered, and the full name of ET is edge-triggered.
Although this knowledge point is very popular, many people's understanding of it is always a bit poor, especially during the interview, many interviewers are always in a state of recalling and reciting. In fact, these two modes are really different. It is necessary to memorize by rote. Let me talk about my personal understanding and memory methods of these two modes.
memory of name
Every time ET (Edge Trigger) is mentioned, the first thing that comes to my mind is the course "Digital Logic Circuit" in the university. It will mention low level and high level. When the level goes from low to high, there will be a Rising edge, and there will be a falling edge when the level goes from high to low. This "edge" is the "edge" mentioned in edge triggering. It is the same concept as the curb on the side of the road, that is, when the state changes. . I was very impressed when I mentioned the rising edge and falling edge. At that time, I took up several lessons and wrote a simulation waveform displaying "HELLO WORLD" in Verilog language, relying on the "edge" in the level change.
state change
LT mode and ET mode can be learned by analogy to level changes, but the concepts are not exactly the same in practical applications. The application of epoll involves reading and writing about IO, and what are the status changes of reading and writing? Readable, unreadable, writable, and unwritable are actually just these four states. Take socket as an example.
Readable: there is data on the socket
Unreadable: There is no data on the socket
Writable: There is space on the socket for writing
Not writable: no space on the socket to write
For the horizontal trigger mode, an event will always be triggered as long as there is one.
For edge-triggered mode, only one event fires from nothing.
LT mode
For read events EPOLLIN, as long as there is unread data on the socket, EPOLLIN it will always be triggered; for write events EPOLLOUT, as long as the socket is writable (one said that the TCP window is not saturated, I think it is when the TCP buffer is not full, this One point needs to be verified), EPOLLOUT it will always be triggered.
In this mode, everyone will think that reading data will be easier, because even if the data has not been read, the next time you call epoll_wait(), it will notify you to continue reading on the file descriptor that has not been read. It is the mode that people often say that there is no need to worry about losing data.
When writing data, because the use of LT mode will always trigger the EPOLLOUT event, then if the code implementation relies on writable event triggers to send data, be sure to remove the detection of writable events after the data is sent to avoid meaninglessness when there is no data to send trigger.
ET mode
For read events , it will only be triggered EPOLLINwhen the data on the socket starts from scratch ; for write events , it will only be triggered when the socket write buffer changes from non-writable to writable ( when the just added event finishes calling epoll_wait or the buffer from full to full)EPOLLINEPOLLOUTEPOLLOUT
This mode sounds much more refreshing. Notifications will only be made when the state changes. Fewer notifications will naturally cause some problems. For example, the data must be collected after the read event is triggered, because you may not have the next chance to collect it again. Even if the data is not read cleanly at one time, the activation status must be recorded and processed later, otherwise, if the data remains until the next message arrives, it will cause delays.
After the write event is triggered in this mode, it will not be triggered again later. If the next write event trigger is needed to drive and send data, you need to register again to detect the writable event.
Data reading and sending
It is easier to understand the reading of data. Whether it is LT mode or ET mode, it is enough to start reading data from the socket after listening to the read event, but the logic of reading is somewhat different. In LT mode, after the read event is triggered, it can be collected on demand. For the desired number of bytes, it is not necessary to collect the data received this time. In ET mode, after the read event is triggered, the data usually needs to be collected at one time.
The writing of data is not easy to understand, because the reading of data is caused by the data sent by the peer, and the writing of data is actually triggered by its own logic layer, so it is usually not registered when sending data through the network. Write events are usually sent directly by calling the send or write function. If the function returns -1 during the sending process, and the error code is EWOULDBLOCK, indicating that the sending failed, then the listener for writable events will be registered and the remaining services will be stored in In the custom send buffer, wait for the writable event to trigger and then send the remaining data in the send buffer.
Related video recommendation
Epoll actual combat secret - the underlying cornerstone supporting billion-level IO
Free learning address: c/c++ linux server development/background architect
Need C/C++ Linux server architect learning materials plus qun 812855908 to obtain (materials include C/C++, Linux, golang technology, Nginx, ZeroMQ, MySQL, Redis, fastdfs, MongoDB, ZK, streaming media, CDN, P2P, K8S, Docker , TCP/IP, coroutine, DPDK, ffmpeg, etc.), free to share
code practice
base code
The following is the basic code of an epoll trigger mode test, which is not too long and can be tested directly:
#include <sys/socket.h> //for socket
#include <arpa/inet.h> //for htonl htons
#include <sys/epoll.h> //for epoll_ctl
#include <unistd.h> //for close
#include <fcntl.h> //for fcntl
#include <errno.h> //for errno
#include <iostream> //for cout
class fd_object
{
public:
fd_object(int fd) { listen_fd = fd; }
~fd_object() { close(listen_fd); }
private:
int listen_fd;
};
/*
./epoll for lt mode
and
./epoll 1 for et mode
*/
int main(int argc, char* argv[])
{
//create a socket fd
int listen_fd = socket(AF_INET, SOCK_STREAM, 0);
if (listen_fd == -1)
{
std::cout << "create listen socket fd error." << std::endl;
return -1;
}
fd_object obj(listen_fd);
//set socket to non-block
int socket_flag = fcntl(listen_fd, F_GETFL, 0);
socket_flag |= O_NONBLOCK;
if (fcntl(listen_fd, F_SETFL, socket_flag) == -1)
{
std::cout << "set listen fd to nonblock error." << std::endl;
return -1;
}
//init server bind info
int port = 51741;
struct sockaddr_in bind_addr;
bind_addr.sin_family = AF_INET;
bind_addr.sin_addr.s_addr = htonl(INADDR_ANY);
bind_addr.sin_port = htons(port);
if (bind(listen_fd, (struct sockaddr *)&bind_addr, sizeof(bind_addr)) == -1)
{
std::cout << "bind listen socket fd error." << std::endl;
return -1;
}
//start listen
if (listen(listen_fd, SOMAXCONN) == -1)
{
std::cout << "listen error." << std::endl;
return -1;
}
else
std::cout << "start server at port [" << port << "] with [" << (argc <= 1 ? "LT" : "ET") << "] mode." << std::endl;
//create a epoll fd
int epoll_fd = epoll_create(88);
if (epoll_fd == -1)
{
std::cout << "create a epoll fd error." << std::endl;
return -1;
}
epoll_event listen_fd_event;
listen_fd_event.data.fd = listen_fd;
listen_fd_event.events = EPOLLIN;
if (argc > 1) listen_fd_event.events |= EPOLLET;
//add epoll event for listen fd
if (epoll_ctl(epoll_fd, EPOLL_CTL_ADD, listen_fd, &listen_fd_event) == -1)
{
std::cout << "epoll ctl error." << std::endl;
return -1;
}
while (true)
{
epoll_event epoll_events[1024];
int n = epoll_wait(epoll_fd, epoll_events, 1024, 1000);
if (n < 0)
break;
else if (n == 0) //timeout
continue;
for (int i = 0; i < n; ++i)
{
if (epoll_events[i].events & EPOLLIN)//trigger read event
{
if (epoll_events[i].data.fd == listen_fd)
{
//accept a new connection
struct sockaddr_in client_addr;
socklen_t client_addr_len = sizeof(client_addr);
int client_fd = accept(listen_fd, (struct sockaddr*)&client_addr, &client_addr_len);
if (client_fd == -1)
continue;
socket_flag = fcntl(client_fd, F_GETFL, 0);
socket_flag |= O_NONBLOCK;
if (fcntl(client_fd, F_SETFL, socket_flag) == -1)
{
close(client_fd);
std::cout << "set client fd to non-block error." << std::endl;
continue;
}
epoll_event client_fd_event;
client_fd_event.data.fd = client_fd;
client_fd_event.events = EPOLLIN | EPOLLOUT;
if (argc > 1) client_fd_event.events |= EPOLLET;
if (epoll_ctl(epoll_fd, EPOLL_CTL_ADD, client_fd, &client_fd_event) == -1)
{
std::cout << "add client fd to epoll fd error." << std::endl;
close(client_fd);
continue;
}
std::cout << "accept a new client fd [" << client_fd << "]." << std::endl;
}
else
{
std::cout << "EPOLLIN event triggered for client fd [" << epoll_events[i].data.fd << "]." << std::endl;
char recvbuf[1024] = { 0 };
int m = recv(epoll_events[i].data.fd, recvbuf, 1, 0); // only read 1 bytes when read event triggered
if (m == 0 || (m < 0 && errno != EWOULDBLOCK && errno != EINTR))
{
if (epoll_ctl(epoll_fd, EPOLL_CTL_DEL, epoll_events[i].data.fd, NULL) != -1)
std::cout << "the client fd [" << epoll_events[i].data.fd << "] disconnected." << std::endl;
close(epoll_events[i].data.fd);
}
std::cout << "recv data from client fd [" << epoll_events[i].data.fd << "] and data is [" << recvbuf << "]." << std::endl;
}
}
else if (epoll_events[i].events & EPOLLOUT)
{
if (epoll_events[i].data.fd == listen_fd) //trigger write event
continue;
std::cout << "EPOLLOUT event triggered for client fd [" << epoll_events[i].data.fd << "]." << std::endl;
}
}
}
return 0;
}Briefly speaking, the test method of this code can g++ testepoll.cpp -o epoll be compiled. After compiling, ./epoll it runs in LT mode and ./epoll etET mode. We use the compiled epoll program as the server and use the nc command to simulate a client.
Test classification
1. Directly ./epoll after compiling, and then use the nc -v 127.0.0.1 51741 command to simulate a connection in another command line window. At this time, ./epoll will generate a large number of EPOLLOUT event triggered for client fd ..., that is Because in LT mode, EPOLLOUT will always be triggered.
albert@home-pc:/mnt/d/data/cpp/testepoll$ ./epoll
start server at port [51741] with [LT] mode.
accept a new client fd [5].
EPOLLOUT event triggered for client fd [5].
EPOLLOUT event triggered for client fd [5].
EPOLLOUT event triggered for client fd [5].
EPOLLOUT event triggered for client fd [5].
EPOLLOUT event triggered for client fd [5].
EPOLLOUT event triggered for client fd [5].
EPOLLOUT event triggered for client fd [5].
EPOLLOUT event triggered for client fd [5].
...2. Comment out the 152nd line of code that contains the output of EPOLLOUT event triggered for client fd. After compiling, run ./epoll, and then use nc -v 127.0.0.1 51741 to simulate a connection in another command line window, and then enter abcd and press Enter. You can see the output of the server ./epoll, EPOLLIN is triggered multiple times, and one byte is read each time.
albert@home-pc:/mnt/d/data/cpp/testepoll$ ./epoll
start server at port [51741] with [LT] mode.
accept a new client fd [5].
EPOLLIN event triggered for client fd [5].
recv data from client fd [5] and data is [a].
EPOLLIN event triggered for client fd [5].
recv data from client fd [5] and data is [b].
EPOLLIN event triggered for client fd [5].
recv data from client fd [5] and data is [c].
EPOLLIN event triggered for client fd [5].
recv data from client fd [5] and data is [d].
EPOLLIN event triggered for client fd [5].
recv data from client fd [5] and data is [
].3. Restore the line of code just commented, compile and execute to start ./epoll et the server, and then simulate a connection in another command line window nc -v 127.0.0.1 51741 , and then simulate a connection in another command line window , the server window shows that the event nc -v 127.0.0.1 51741 is triggeredEPOLLOUT
albert@home-pc:/mnt/d/data/cpp/testepoll$ ./epoll et
start server at port [51741] with [ET] mode.
accept a new client fd [5].
EPOLLOUT event triggered for client fd [5].On this basis, ncenter a carriage return, enter a carriage return, and output a carriage return from the window that just ran the command, then the epoll server window sees that three events are triggered EPOLLIN, and each time a carriage return is received:
albert@home-pc:/mnt/d/data/cpp/testepoll$ ./epoll et
start server at port [51741] with [ET] mode.
accept a new client fd [5].
EPOLLOUT event triggered for client fd [5].
EPOLLIN event triggered for client fd [5].
recv data from client fd [5] and data is [
].
EPOLLIN event triggered for client fd [5].
recv data from client fd [5] and data is [
].
EPOLLIN event triggered for client fd [5].
recv data from client fd [5] and data is [
].But if you output abcd and press Enter in the nc simulated client, then the EPOLLIN event will be triggered once in the epoll server window and EPOLLIN will not be triggered after receiving an a, even if you enter it here on the nc client, it is useless. That's because there is still data in the received buffer, and when new data comes, the buffer does not change from empty to data, so pay attention to this situation in ET mode.
albert@home-pc:/mnt/d/data/cpp/testepoll$ ./epoll et
start server at port [51741] with [ET] mode.
accept a new client fd [5].
EPOLLOUT event triggered for client fd [5].
EPOLLIN event triggered for client fd [5].
recv data from client fd [5] and data is [
].
EPOLLIN event triggered for client fd [5].
recv data from client fd [5] and data is [
].
EPOLLIN event triggered for client fd [5].
recv data from client fd [5] and data is [
].
EPOLLIN event triggered for client fd [5].
recv data from client fd [5] and data is [a].How to solve the problem that ET triggers once and then no longer triggers
Change the code, the ET mode is triggered once after the connection EPOLLOUT, and once when the data is received EPOLLIN. If the data is confiscated, these two events will never be triggered again. If you want to change this situation, you can register these two events again. For an event, the timing can be selected when the data is received, so you can modify this part of the code:
else
{
std::cout << "EPOLLIN event triggered for client fd [" << epoll_events[i].data.fd << "]." << std::endl;
char recvbuf[1024] = { 0 };
int m = recv(epoll_events[i].data.fd, recvbuf, 1, 0); // only read 1 bytes when read event triggered
if (m == 0 || (m < 0 && errno != EWOULDBLOCK && errno != EINTR))
{
if (epoll_ctl(epoll_fd, EPOLL_CTL_DEL, epoll_events[i].data.fd, NULL) != -1)
std::cout << "the client fd [" << epoll_events[i].data.fd << "] disconnected." << std::endl;
close(epoll_events[i].data.fd);
}
std::cout << "recv data from client fd [" << epoll_events[i].data.fd << "] and data is [" << recvbuf << "]." << std::endl;
}Add the logic to register again:
else
{
std::cout << "EPOLLIN event triggered for client fd [" << epoll_events[i].data.fd << "]." << std::endl;
char recvbuf[1024] = { 0 };
int m = recv(epoll_events[i].data.fd, recvbuf, 1, 0); // only read 1 bytes when read event triggered
if (m == 0 || (m < 0 && errno != EWOULDBLOCK && errno != EINTR))
{
if (epoll_ctl(epoll_fd, EPOLL_CTL_DEL, epoll_events[i].data.fd, NULL) != -1)
std::cout << "the client fd [" << epoll_events[i].data.fd << "] disconnected." << std::endl;
close(epoll_events[i].data.fd);
}
epoll_event client_fd_event;
client_fd_event.data.fd = epoll_events[i].data.fd;
client_fd_event.events = EPOLLIN | EPOLLOUT;
if (argc > 1) client_fd_event.events |= EPOLLET;
epoll_ctl(epoll_fd, EPOLL_CTL_MOD, epoll_events[i].data.fd, &client_fd_event);
std::cout << "recv data from client fd [" << epoll_events[i].data.fd << "] and data is [" << recvbuf << "]." << std::endl;
}This time, ./epoll etstart the server in mode, use nc -v 127.0.0.1 51741the simulated client, enter abc and press Enter to find that the output of the epoll server shows that the triggered event has changed:
albert@home-pc:/mnt/d/data/cpp/testepoll$ ./epoll et
start server at port [51741] with [ET] mode.
accept a new client fd [5].
EPOLLOUT event triggered for client fd [5].
EPOLLIN event triggered for client fd [5].
recv data from client fd [5] and data is [a].
EPOLLIN event triggered for client fd [5].
recv data from client fd [5] and data is [b].
EPOLLIN event triggered for client fd [5].
recv data from client fd [5] and data is [c].
EPOLLIN event triggered for client fd [5].
recv data from client fd [5] and data is [
].
EPOLLOUT event triggered for client fd [5].Summarize
- LT mode will always trigger EPOLLOUT, and will always trigger EPOLLIN when there is data in the buffer
- ET mode will trigger an EPOLLOUT after the connection is established, and an EPOLLIN will be triggered when data is received
- When the LT mode triggers EPOLLIN, the data can be read on demand, and the remaining data will be notified to read again
- When ET mode triggers EPOLLIN, the data must be read, otherwise even if new data comes, it will not be notified again
- EPOLLOUT in LT mode will always be triggered, so remember to delete the data after sending, otherwise a lot of unnecessary notifications will be generated
- In the EPOLLOUT event of ET mode, if the data has not been sent, it needs to be registered again, otherwise there will be no chance to send it again
- Usually, the EPOLLOUT event will not be relied on when sending network data. This event will only be registered when the buffer is full and fails to send, and it is expected to be sent again after being notified.