3. The creation of epoll and time wheel
When creating the thread environment, an epoll structure will be created for the thread environment for the scheduling of the coroutine. Next, let's find out.
void co_init_curr_thread_env()
{
... ...
stCoEpoll_t *ev = AllocEpoll(); // 为线程环境创建一个epoll的结构
SetEpoll(env, ev);
}1. Introduction to Time Wheel
The time wheel is an algorithm used to implement a timer.
The time wheel is composed of multiple time grids, and each time grid represents the basic time span (tickMs) of the current time wheel. The number of time grids of the time wheel is fixed and can be represented by wheelSize. Then the overall time span (interval) of the entire time wheel can be calculated by the formula tickMs × wheelSize. The time wheel also has a dial pointer (currentTime), which is used to indicate the current time of the time wheel, and currentTime is an integer multiple of tickMs. CurrentTime can divide the entire time wheel into an expiration part and an unexpired part. The time grid currently pointed to by currentTime also belongs to the expiration part, which means that it just expires and needs to process all tasks of the linked list corresponding to this time grid.
If tickMs=1ms and wheelSize=20 for the time wheel, then the interval can be calculated to be 20ms. In the initial situation, the dial pointer currentTime points to time grid 0. At this time, a task with a timing of 2ms is inserted and stored in the linked list with time grid 2. As time goes by, the pointer currentTime keeps moving forward. After 2ms, when the time grid 2 is reached, the task in the linked list corresponding to the time grid 2 needs to be expired accordingly. At this time, if another task with a timing of 8ms is inserted in, it will be stored in time grid 10, and currentTime will point to time grid 10 after another 8ms. What if a task with a timing of 19ms is inserted at the same time? What if there is a task with a timing of 350ms at this time? When the task's due time exceeds the time range indicated by the current time wheel, it will try to add it to the upper time wheel.
Refer to the figure above, the case before reuse, the time wheel of the first layer is tickMs=1ms, wheelSize=20, interval=20ms. The tickMs of the second-level time wheel is the interval of the first-level time wheel, which is 20ms. The wheelSize of each time wheel is fixed at 20, so the overall time span of the second time wheel is 400ms. By analogy, this 400ms is also the size of the third layer's tickMs, and the overall time span of the third layer's time wheel is 8000ms.
For the aforementioned 350ms timing task, it is obvious that the first-level time wheel cannot meet the conditions, so it is upgraded to the second-level time wheel, and finally inserted into the linked list corresponding to the time grid 17 in the second-level time wheel. If there is another task with a timing of 450ms at this time, then obviously the second-level time wheel cannot meet the conditions, so it is upgraded to the third-level time wheel and finally inserted into the linked list of time grid 1 in the third-level time wheel. in. Note that multiple tasks (such as 446ms, 455ms, and 473ms timed tasks) whose expiration time is in the interval of [400ms, 800ms) will be put into the time grid 1 of the third layer of time wheel, and the linked list corresponding to time grid 1 The timeout period is 400ms. With the passage of time, when the secondary linked list expires, the task originally scheduled for 450ms has 50ms left, and the expiration operation of this task cannot be executed. Here is a downgrade operation of the time wheel, which will resubmit this timed task with a remaining time of 50ms to the level time wheel. At this time, the overall time span of the first level time wheel is not enough, and the second level is enough, so the task It is placed in the time grid with the expiration time of the second layer of time wheel [40ms, 60ms). After another 40ms, the task was "perceived" again at this time, but there was still 10ms left, and the expiration operation could not be executed immediately. So there will be another downgrade of the time wheel. This task is added to the time grid where the expiration time of the first time wheel is [10ms,11ms). After another 10ms, the task really expires, and the corresponding task is finally executed. The expiration operation.
2. Epoll and time wheel
struct stCoEpoll_t
{
int iEpollFd; // epoll 专用的文件描述符
static const int _EPOLL_SIZE = 1024 * 10; // epoll可以监听的事件数，默认为10240，这是一个静态变量
struct stTimeout_t *pTimeout; // 单轮时间轮，相当于一个定时器
struct stTimeoutItemLink_t *pstTimeoutList; // 链表用于临时存放超时事件的item
struct stTimeoutItemLink_t *pstActiveList; // 该链表用于存放epoll_wait得到的就绪事件和定时器超时事件
co_epoll_res *result; // 对 epoll_wait()第二个参数的封装，即一次 epoll_wait 得到的结果集
};
struct co_epoll_res
{
int size;
struct epoll_event *events;
struct kevent *eventlist;
};
/*
* 毫秒级的超时管理器，即一个定时器的实现
* 使用时间轮实现
* 但是是有限制的，最长超时时间不可以超过iItemSize毫秒
*/
struct stTimeout_t
{
/*
* 为一个链表，超时事件数组，总长度为iItemSize,每一项代表1毫秒，代表这个时间所超时的事件。
* 这个数组在使用的过程中，会使用取模的方式，把它当做一个循环数组来使用
*/
stTimeoutItemLink_t *pItems;
int iItemSize; // 数组长度，也即最长的定时时间，单位为ms
unsigned long long ullStart; // 时间轮第一次使用的时间
long long llStartIdx; // 目前正在使用的下标
};
struct stTimeoutItemLink_t
{
stTimeoutItem_t *head; // 链表头指针
stTimeoutItem_t *tail; // 链表尾指针
};
typedef void (*OnPreparePfn_t)( stTimeoutItem_t *,struct epoll_event &ev, stTimeoutItemLink_t *active );
typedef void (*OnProcessPfn_t)( stTimeoutItem_t *);
struct stTimeoutItem_t
{
enum
{
eMaxTimeout = 40 * 1000 // poll注册事件时的最大超时时间为40s
};
stTimeoutItem_t *pPrev; // 链表节点的前一个节点
stTimeoutItem_t *pNext; // 链表的节点下一个节点
stTimeoutItemLink_t *pLink; // 该节点所属的链表
unsigned long long ullExpireTime; // 超时时间
OnPreparePfn_t pfnPrepare; // 预处理回调函数，在epoll_loop中被调用
OnProcessPfn_t pfnProcess; // 正式的回调函数，在epoll_loop中被调用
void *pArg; // 指向协程实体
bool bTimeout; // 是否超时
};
3. The creation of epoll and time wheel
stCoEpoll_t *AllocEpoll()
{
stCoEpoll_t *ctx = (stCoEpoll_t*)calloc( 1,sizeof(stCoEpoll_t) );
ctx->iEpollFd = co_epoll_create( stCoEpoll_t::_EPOLL_SIZE );
ctx->pTimeout = AllocTimeout( 60 * 1000 ); // 最大超时时间为60s
ctx->pstActiveList = (stTimeoutItemLink_t*)calloc( 1,sizeof(stTimeoutItemLink_t) );
ctx->pstTimeoutList = (stTimeoutItemLink_t*)calloc( 1,sizeof(stTimeoutItemLink_t) );
return ctx;
}
stTimeout_t *AllocTimeout( int iSize )
{
stTimeout_t *lp = (stTimeout_t*)calloc( 1,sizeof(stTimeout_t) );
lp->iItemSize = iSize;
lp->pItems = (stTimeoutItemLink_t*)calloc( 1,sizeof(stTimeoutItemLink_t) * lp->iItemSize );
lp->ullStart = GetTickMS();
lp->llStartIdx = 0;
return lp;
}The time wheel here is implemented using a linked list, which is a single-round time wheel. Generally speaking, when the complexity of a single-round time wheel is reduced, when the timeout time is greater than the time wheel length, it needs to take the remainder and put it in. As a result, there will be some invalid traversals every time the time wheel is taken out. When the time came, he refused directly. However, due to its characteristics, it is difficult for the multi-round time wheel to have a timeout longer than the time wheel length, so there is no invalid traversal, but some copies are required.