Blocking IO model
Blocking IO is the most common type of IO. When using this model for data reception, the program will wait until the data arrives. For example, for the function recvfrom(), the kernel will block the request until data arrives before returning.
Non-blocking IO model
When the socket is set to non-blocking IO, the kernel will not block each request and will return immediately; when there is no data, an error will be returned. For example, for the recvfrom() function, no data is returned for the first few times, and the kernel does not copy data to the user layer space until the end.
The biggest difference between non-blocking operations and blocking operations is that the function call returns immediately, regardless of whether the data is successfully read or written. After setting the fcntl() socket file descriptor according to the following code, non-blocking programming can be performed:
fcntl(s, F_SETFL, O_NONBLOCK);
Among them, s is the socket file descriptor. After using the F_SETFL command to set the socket s to non-blocking mode, it can return immediately after reading and writing operations.
fcntl function prototype:
#include <unistd.h>
#include <fcntl.h>int fcntl(int fd, int cmd, ... /* arg */ );
The fcntl function has 5 functions:
- Duplicate an existing descriptor (cmd=F_DUPFD).
- Get/set file descriptor flags (cmd=F_GETFD or F_SETFD).
- Get/set file status flags (cmd=F_GETFL or F_SETFL).
- Get/set asynchronous I/O ownership (cmd=F_GETOWN or F_SETOWN).
- Get/set record lock (cmd=F_GETLK, F_SETLK or F_SETLKW).
We only use the third function here, getting/setting the file status flag, to set a file descriptor as non-blocking.
#include <iostream>
#include <cerrno>
#include <fcntl.h>
#include <unistd.h>
bool SetNonBlock(int fd)
{
int fl = fcntl(fd,F_GETFD);//在底层获取当前fd对应的文件读写标志位
if(fl < 0)
{
return false;
}
fcntl(fd,F_SETFL,fl | O_NONBLOCK);//设置非阻塞
return true;
}
int main()
{
SetNonBlock(0);
char buffer[1024];
while(true)
{
sleep(1);
errno = 0;
ssize_t s = read(0,buffer,sizeof(buffer)-1);
if(s > 0)
{
buffer[s] = 0;
std::cout << "echo# " << buffer << " errno "<< errno << "errstring: " << std::endl;
}
else
{
//errno的值是11，代表底层数据没就绪
//std::cout << " read \"error\" " << " errno "<< errno << "errstring: " << std::endl;
if(errno == EWOULDBLOCK ||errno == EAGAIN)
{
std::cout << " 当前0号fd数据没有就绪" << std::endl;
continue;
}
else if(errno == EINTR)
{
std::cout << " 当前0号fd数据没有就绪" << std::endl;
continue;
}
}
}
return 0;
}
I/O multiplexing
Using the IO multiplexing model, you can add a timeout time while waiting. When the timeout time is not reached, the blocking situation is the same. When the timeout time is reached and no data is received, the system will return and no longer wait. select() The function polls according to a certain timeout period until the socket that needs to wait for data arrives, and uses the recvfrom() function to copy the data to the application layer.
The essence of IO is waiting + data copying . To make IO more efficient is to shorten the waiting time. The select() function can shorten the waiting time. The select() function can help users wait for multiple file socks at a time. When which file socks are ready, the select() function will notify the user of the corresponding socks, and then the user will call recv/recvfrom/read and other functions to read .
select () function
The function select () is different from the previous functions recv () and send () to directly manipulate the file descriptor. Using the select() function, you can first query the file descriptor that needs to be operated, check whether the target file descriptor can be read, written or wrongly operated, and then perform the real IO operation when the file descriptor meets the operation conditions.
/* According to POSIX.1-2001 */
#include <sys/select.h>/* According to earlier standards */
#include <sys/time.h>
#include <sys/types.h>
#include <unistd.h>int select(int nfds, fd_set *readfds, fd_set *writefds,
fd_set *exceptfds, struct timeval *timeout);
In addition to nfds are input and output parameters. fd_set is a bitmap structure.
The function select() allows the program to monitor multiple file descriptors, and returns when one or more monitored file descriptions are ready for IO operations. The corresponding operation of the function monitoring a file descriptor is not available, for example, the file descriptor of the monitoring read file set is operable.
The function can monitor 3 types of file descriptors at the same time. It will monitor whether the file in the readfds file descriptor set is readable, that is, judge whether the read operation of this file descriptor is blocked; the function monitors whether the file in the writeids file descriptor set is writable, that is, judge whether the file is described Whether the write operation of the character is blocked; in addition, the function also monitors whether the file descriptor in the file descriptor set exceptfds is unexpected. When the function exits, the above collection is changed. When there is no need to monitor a certain file set, the corresponding file set can be set to NULL. If the sum of all file sets is NULL, it means to wait for a period of time.
The type of parameter timeout is the following structure:
struct timeval
{ time_t tv_ sec; /* sec */
long tv usec; /* microseconds */
};
- The member tv_sec indicates the timeout in seconds.
- The member tv_usec represents the number of microseconds for timeout, which is 1/1000000s.
There are 4 macros that can manipulate collections of file descriptors.
- FD_ZERO(): Clean up the file descriptor collection.
- FD_SET(): Add a file descriptor to a file descriptor set.
- FD_CLR(): Take a certain file descriptor from a collection of certain file descriptors.
- FD_ISSET(): Tests whether a file descriptor is a member of a set.
Also note: There is a maximum limit on the set of file descriptors, the maximum value is FD_SETSIZE, when the maximum value is exceeded, undetermined things will happen.
readfds:
a. When inputting: user->kernel, among my bits, the position of the bit indicates the value of the file descriptor, and the content of the bit indicates whether the user cares.
b. When outputting: Kernel->User, I am the OS, and the multiple fds that the user asked me to care about have results. The position of the bit indicates the value of the file descriptor, and the content of the bit indicates whether it is ready.
Subsequent users can directly read the number represented by the 1 bit in the bitmap without being blocked.
Because both the user and the kernel will modify the same bitmap structure, after this parameter is used once, it must be reset.
The meanings of the parameters of the function select() are as follows.
- nfds: An integer variable that is 1 greater than the maximum value of all file descriptors in the file descriptor set. When using select(), the value of the file description of the maximum value must be calculated, and the value is passed in through nfds.
- readfds: This file descriptor set monitors whether any files in the file set have readable data. When the select() function returns, readfds will clear the unreadable file descriptors, leaving only readable file descriptors, that is It can be read data by functions recv(), read(), etc.
- writefds: This file descriptor set monitors whether any files in the file set have data to write. When the select() function returns, readfds will clear the non-writable file descriptors, leaving only writable file descriptors. That is, it can be used to write data by send(), write() functions, etc.
- exceptfds: This file set will monitor any files in the file set for errors. In fact, it can be used for other purposes. For example, to monitor out-of-band data OOB, out-of-band data is sent to the socket using the MSG_OOB flag. When the select() function returns, readfds will clear other file descriptors, leaving only readable OOB data.
- timeout: Set the longest waiting time when the events in the file collection monitored by select() do not occur. When this time is exceeded, the function will return. When the timeout is NULL, it means a blocking operation, and it will wait until a monitored file set is 0, and selet will return immediately. (Select waits for multiple fds, and the waiting strategy can be selected: 1. Blocking nullptr 2. Non-blocking {0, 0} 3. You can set the timeout time, block within the time, and return immediately {(5,0)} when the time is up
During the waiting time, if there is fd ready, timeout will show output, and return the remaining time from the next timeout.)
The return value of the function select() is 0, -1 or an integer value greater than 1: when there is a file descriptor in the monitored file set that meets the requirements, that is, the files in the read file descriptor set can be read, and the files in the write file descriptor can be read. When an error occurs in the write or error file descriptor, the return value is a positive value greater than 0; when it times out, it returns 0; when an error occurs when the return value is -1, the error value is specified by errno.
Error values may be:
EBADF: The file descriptor is invalid or the file is closed
EINTR: This call was interrupted by a signal
EINVAL: An invalid parameter was passed
ENOMEM : not enough memory
Select Advantages and Disadvantages:
Advantages: Any multi-channel transfer scheme has:
- efficient
- Application scenario: There are a large number of connections, but only a small number are active, saving resources
shortcoming:
- In order to maintain the third-party array, the select server will be filled with a large number of traversals. When the bottom layer of the OS helps us care about fd, it must also traverse b. The select output parameters must be reset every time.
- There is an upper limit to the number of fds that can be managed at the same time
- Because almost every parameter is input and output, select will frequently copy the parameter data from user to kernel and from kernel to user
- Coding is more complicated
poll() function
The poll() function waits for an event to occur on a file descriptor
#include <poll.h>
int poll(struct pollfd *fds, nfds_t nfds, int timeout);
The poll() function monitors the actions that occur on a set of file descriptors specified by the fds array, and exits when the conditions are met or the packet times out.
- The parameter fds is a pointer to the structure poll array, and the monitored file descriptors and conditions are placed in it.
- The parameter nfds is a value 1 greater than the value of the largest descriptor to monitor.
- The parameter timeout is the timeout time in milliseconds. When it is negative, it means waiting forever.
The meaning of the return value of the poll() function is as follows.
Greater than 0: Indicates success, a waiting condition is met, and the return value is the number of monitoring file descriptors that meet the condition. 0: means timeout. -1: Indicates that an error occurred, and the error code of errno is the same as that of select.
The prototype of the structure struct poll is as follows:
struct pollfd { int fd; /*file descriptor*/ short events; /*requested events*/ short revents; /*returned events*/ };
- The member fd represents the file descriptor to monitor.
- The member events represents the input monitoring event, and its value and meaning are as follows.
- The member revents indicates the returned monitoring event, that is, the event that occurs when returning.
Advantages of poll:
- efficient
- There are a large number of connections, but only a small number are active. saves resources
- The input and output parameters are separated and do not need to be reset extensively.
- poll parameter level, there is no upper limit of fd that can be managed
Poll Disadvantages:
- poll still needs a lot of traversal, the user layer detects that the time is ready, and the kernel detects that fd is ready. All the same, the user still has to maintain the array
- poll requires a kernel-to-user copy.
- The poll code is also more complicated - easier than select.
epoll() function
epoll has 3 related system calls.
epoll_create()
Create a handle to epoll
#include <sys/epoll.h>
int epoll_create(int size);
- Since linux2.6.8, the size parameter is ignored.
- When you are done with it, you must call close() to close it.
epoll_ctl()
Event registration function of epoll
#include <sys/epoll.h>
int epoll_ctl(int epfd, int op, int fd, struct epoll_event *event);
It is different from select(), which tells the kernel what type of event to listen to when listening to the event, but first registers the type of event to be listened to here. The first parameter is the return value of epoll_create() (the handle of epoll); the second parameter represents the action, represented by three macros; the third parameter is the fd that needs to be monitored; the fourth parameter is to tell the kernel to Listen for something.
The value of the second parameter:
- EPOLL_CTL_ADD: Register a new fd to epfd;
- EPOLL_CTL_MOD: Modify the listening event of the registered fd;
- EPOLL_CTL_DEL: delete a fd from epfd;
struct epoll_event structure
typedef union epoll_data
{
void *ptr;int fd;
uint32_t u32;uint64_t u64;
}epoll_data_t;
struct epoll_event{
uint32_t events;epoll_data_t data;
}_EPOLL_PACKED;
events can be a collection of the following macros:
- EPOLLIN: Indicates that the corresponding file descriptor can be read (including the normal closing of the peer SOCKET);
- EPOLLOUT: Indicates that the corresponding file descriptor can be written;
- EPOLLPRI: Indicates that the corresponding file descriptor has urgent data readable (here it should indicate that there is out-of-band data coming); EPOLLERR: Indicates that the corresponding file descriptor has an error;
- EPOLLHUP: Indicates that the corresponding file descriptor is hung up;
- EPOLLET : Set EPOLL to Edge Triggered mode, which is relative to Level Triggered.
- EPOLLONESHOT: Only listen to one event. After listening to this event, if you need to continue to monitor this socket, you need to add this socket to the EPOLL queue again.
epoll_wait()
Collect the events that have been sent in the events monitored by epoll.
#include <sys/epoll.h>
int epoll_wait(int epfd, struct epoll_event *events,
int maxevents, int timeout);
The parameter events is the allocated epoll_event structure array. epoll will assign the events that occurred to the events array (events cannot be a null pointer, the kernel is only responsible for copying data to the events array, and will not help us in the user state). maxevents tells the kernel how big this event is, and the value of this maxevents cannot be greater than the size when creating epoll_create(). The parameter timeout is the timeout time (milliseconds, 0 will return immediately, -1 is permanent blocking). If If the function is called successfully, it returns the number of prepared file descriptors on the corresponding I/O. If it returns 0, it means it has timed out. If it returns less than 0, it means the function failed.
How the epoll() function works
epoll.hpp
#include <iostream>
#include <sys/epoll.h>
#include <unistd.h>
class Epoll
{
public:
static const int gsize = 256;
public:
static int CreateEpoll()
{
int epfd = epoll_create(gsize);
if(epfd > 0) return epfd;
exit(5);
}
static bool CtlEpoll(int epfd, int oper, int sock, uint32_t events)
{
struct epoll_event ev;
ev.events = events;
ev.data.fd = sock;
int n = epoll_ctl(epfd, oper, sock, &ev);
return n == 0;
}
static int WaitEpoll(int epfd, struct epoll_event revs[], int num, int timeout)
{
//如果底层就绪的sock非常多，revs承装不下，一次拿不完，就下一次再拿
//关于epoll_wait的返回值问题：有多少个fd上的事件就绪，就返回多少，epoll返回的时候，
//会将所有就绪的event按照顺序放入到revs数组中，一共有返回值个
return epoll_wait(epfd, revs, num, timeout);
}
};epollServer.hpp
#ifndef __EPOLL_SERVER_HPP__
#define __EPOLL_SERVER_HPP__
#include <iostream>
#include <string>
#include <functional>
#include <cassert>
#include "Log.hpp"
#include "Sock.hpp"
#include "Epoll.hpp"
namespace ns_epoll
{
const static int default_port = 8080;
const static int gnum = 64;
//只处理读取
class EpollServer
{
using func_t = std::function<void(std::string)>;
public:
EpollServer(func_t HandlerRequest, const int &port = default_port)
: _port(port), _revs_num(gnum), _HandlerRequest(HandlerRequest)
{
// 0. 申请对应的空间
_revs = new struct epoll_event[_revs_num];
// 1. 创建listensock
_listensock = Sock::Socket();
Sock::Bind(_listensock, _port);
Sock::Listen(_listensock);
// 2. 创建epoll模型
_epfd = Epoll::CreateEpoll();
logMessage(DEBUG, "init success, listensock: %d, epfd: %d", _listensock, _epfd); // 3, 4
// 3. 将listensock，先添加到epoll中，让epoll帮我们管理起来
if (!Epoll::CtlEpoll(_epfd, EPOLL_CTL_ADD, _listensock, EPOLLIN))
exit(6);
logMessage(DEBUG, "add listensock to epoll success."); // 3, 4
}
void Accepter(int listensock)
{
std::string clientip;
uint16_t clientport;
int sock = Sock::Accept(listensock, &clientip, &clientport);
if(sock < 0)
{
logMessage(WARNING, "accept error!");
return;
}
// 不能直接读取,因为并不清楚，底层是否有数据
// 将新的sock，添加给epoll
if (!Epoll::CtlEpoll(_epfd, EPOLL_CTL_ADD, sock, EPOLLIN)) return;
logMessage(DEBUG, "add new sock : %d to epoll success", sock);
}
void Recver(int sock)
{
// 1. 读取数据
char buffer[10240];
ssize_t n = recv(sock, buffer, sizeof(buffer)-1, 0);
if(n > 0)
{
//假设这里就是读到了一个完整的报文
buffer[n] = 0;
_HandlerRequest(buffer); // 2. 处理数据
}
else if(n == 0)
{
// 1. 先在epoll中去掉对sock的关心
bool res = Epoll::CtlEpoll(_epfd, EPOLL_CTL_DEL, sock, 0);
assert(res);
(void)res;
// 2. 在close文件
close(sock);
logMessage(NORMAL, "client %d quit, me too...", sock);
}
else
{
// 1. 先在epoll中去掉对sock的关心
bool res = Epoll::CtlEpoll(_epfd, EPOLL_CTL_DEL, sock, 0);
assert(res);
(void)res;
// 2. 在close文件
close(sock);
logMessage(NORMAL, "client recv %d error, close error sock", sock);
}
}
void HandlerEvents(int n)
{
assert(n > 0);
for(int i = 0; i < n; i++)
{
uint32_t revents = _revs[i].events;
int sock = _revs[i].data.fd;
// 读事件就绪
if(revents & EPOLLIN)
{
if(sock == _listensock) Accepter(_listensock);
else Recver(sock);
}
if(revents & EPOLLOUT)
{
//TODO?
}
}
}
void LoopOnce(int timeout)
{
int n = Epoll::WaitEpoll(_epfd, _revs, _revs_num, timeout);
//if(n == _revs_num) //扩容
switch (n)
{
case 0:
logMessage(DEBUG, "timeout...");
break;
case -1:
logMessage(WARNING, "epoll wait error: %s", strerror(errno));
break;
default:
// 等待成功
logMessage(DEBUG, "get a event");
HandlerEvents(n);
break;
}
}
void Start()
{
int timeout = -1;
while(true)
{
LoopOnce(timeout);
}
}
~EpollServer()
{
if (_listensock >= 0)
close(_listensock);
if (_epfd >= 0)
close(_epfd);
if (_revs)
delete[] _revs;
}
private:
int _listensock;
int _epfd;
uint16_t _port;
struct epoll_event *_revs;
int _revs_num;
func_t _HandlerRequest;
};
}
#endif
Signal Driven IO Model
Signal-driven IO registers a callback function for signal processing at the beginning of the process, and the process continues to execute. When the signal occurs, there is time for IO, and data arrives here. Use the registered callback function to use recvfrom ()received.
Asynchronous IO model
Asynchronous IO is similar to the previous signal-driven IO, the difference is that the signal-driven IO makes the signal notify the registered signal processing function when the data arrives, while the asynchronous IO sends a signal to notify the registered signal processing function when the data copy is completed. .