muduo

muduo

overview

muduo is a network library based on Reactor mode, which is used to respond to timers and IO events.

Muduo adopts an object-based rather than object-oriented design style, and its event callback adopts function+bind, and users do not need to inherit the class when using muduo

architecture

Multiple Reactor

Reactor mode: There is a cyclic process (IO multiplexing), constantly monitoring whether the corresponding event is triggered, and calling the corresponding callback for processing when the event is triggered

Register the event to the Reactor, and the Reactor will add the corresponding event to the event dispatcher, and the event dispatcher will listen, and if the event occurs, it will tell the Reactor, and the Reactor will call the corresponding event handler

Each Reactor is a thread, and MainReactor is the main IO thread. The listening socket is added into it, and it is only responsible for accepting the client connection. The returned connected socket is added to the subReactor through the algorithm to be monitored and processed.

Threading model: one loop per thread + thread pool

Transform the problem of multi-threaded server-side programming into how to design an efficient and easy-to-use event loop, and then let each thread run an event loop, which can ensure that the resources owned by eventLoop are thread-safe

For example, each TcpConnection (client connection) in muduo has its own buffer. If TcpConnection can be operated by other loops, the buffer will be a critical resource, and thread safety needs to be ensured during operation; if one loop per thread is used , each thread has at most one EventLoop, and each TcpConnection is managed by a certain EventLoop, and all IO will be transferred to this thread, ensuring the thread safety of TcpConnection

Summarize

Muduo adopts the form of Multiple Reactor + ThreadPool: Multiple Reactor is composed of master-slave Reactor, Main Reactor is only responsible for monitoring new connections, after accepting, new connections will be allocated to Sub Reactor, and Sub Reactor is responsible for connection event processing; thread pool Two queues are maintained in it, the task queue and the thread queue. External threads add tasks to the task queue. If the thread queue is not empty, one of the threads will be awakened to process the task (equivalent to the producer and consumer model)

code module

Buffer

Why application layer buffer is necessary in non-blocking network programming

TcpConnection must have output buffer

• For example, the application wants to send 100KB of data, but the operating system only sends out 80KB in the write call, leaving 20KB of data to send
  • If there is no buffer at this time, you need to wait for the operating system to send the remaining 20KB, which will cause blocking
  • But if there is a buffer, the application can only be responsible for generating data, and does not need to care about how the data is sent out. These can be taken care of by the network library. The network library will take over the remaining 20KB of data and save it in the output buffer of the TcpConnection and master-slave the POLLOUT event. When the socket becomes writable, it can send data, and stop paying attention to POLLOUT when the data is sent.

TcpConnection must have input buffer

• TCP is an unbounded byte stream protocol. The receiver must deal with the situation that the received data does not constitute a complete message and the data of two messages is received at a time.
  • In the case of incomplete data, if there is no buffer, it needs to be blocked and wait to receive data
  • If there is a buffer, you can store the data in the buffer first, and wait for the data to be read again to read until a complete message is formed before notifying the program

In general, in order to avoid our program blocking, the input and output buffer is introduced

How to set the buffer size

• System calls are involved in memory allocation, and a relatively large buffer should be prepared
• If the buffer is too large but the space used by each connection is small, the buffer usage will be low.

Use temporary stack space to avoid memory waste caused by too large initial Buffer, and avoid the overhead of repeatedly calling read

• Prepare a 64K extrabuf on the stack, and then use readv() to read the data and store it in iovec. The iovec has two pieces. The first piece points to the writeable byte in the Buffer, and the other piece points to the extrabuf on the stack.
• If there is not much data to read, then all of them are read into the Buffer
• If the length exceeds the writable bytes of the Buffer, it will be read into the extrabuf on the stack, and then append the data in the extrabuf to the Buffer again

For LT mode, only one read is required, and there is no need to call read repeatedly until EAGAIN is returned; for programs that pursue low latency, it is more efficient, and only one system call is required for each read data, while taking care of the fairness of multiple connections Reliability, it will not affect other connections to process messages due to the large amount of data on one connection

How to expand the buffer space is not enough

Is it thread safe

The onMessage callback of the input buffer always occurs in the IO thread to which the TcpConnection belongs. The operation on the buffer is completed in the IO thread to which it belongs, and the buffer is not exposed to other threads to ensure that the buffer is thread-safe.

The output buffer needs to be exposed to other threads. For example, the heartbeat detection thread will use send to send messages to the client. The output buffer needs to be used, which ensures that the buffer is thread-safe through assertInLoopThread

• If TcpConnection::send() sends the IO thread to which TcpConnection belongs, it will call TcpConnection::sendInLoop(), which will operate the output buffer in the current thread
• If it occurs in other threads, the sendInLoop call is transferred to the IO thread to which TcpConnection belongs through EventLoop::runInLoop

That is to say, the buffer will only be operated on the thread to which TcpConnection belongs, and will not be operated on other threads to ensure thread safety

EventLoop

How to guarantee one loop per thread

Thread-local storage allows each thread to have an instance of a variable that can be used to store EventLoop

The constructor of EventLoop will check whether the current thread has created other EventLoop objects, if not, give the current loop to the thread, otherwise the program will terminate if an error occurs

EventLoop::loop

Call Poller::poll to obtain the channel list of the current active event through activeChannels, and then call the handleEvent of each channel in turn to execute the corresponding callback

EventLoop::runInLoop

There are some member functions of EventLoop that allow cross-thread calls. For example, executing a connection timeout callback in an IO thread will cause thread safety issues. Muduo does not use locking, but transfers the operation of the timeout callback to the EventLoop to which the connection belongs. in execution

If the user calls in the current IO thread, the callback will be performed synchronously, otherwise cb will be added to the queue, and the IO thread will be woken up to call this cb

EventLoop::queueInLoop

pendingFunctors can be operated by multiple threads so they need to be locked

The IO thread is usually blocked in the poll call of the event loop EventLoop::loop. In order to allow the IO thread to execute the callback immediately, it needs to wake up the IO thread

How to wake up the IO thread

• Use pipe to let the IO thread always monitor the readable events of the pipe. When it needs to wake up, other threads write a byte to the pipe, and the IO thread will return from the IO multiplexing blocking call
• eventfd is similar to pipeline, but more efficient than pipeline
  • Use one file descriptor less than pipe, saving resources
  • The buffer management is relatively simple, only 8 bytes, and the pipe may have a real buffer of variable length
  • The buffer of eventfd is a 64-bit counter, write the increment counter, read the value of the counter and clear it

When initializing EventLoop, create a wakefd and initialize its corresponding wakeupChannel to handle readable events on wakeupFd, and distribute events to its handleRead function

EventLoop::doPendingFunctors

Swap the callback list into the local variable functors

• Reduce the length of the critical section without blocking other threads calling queueInLoop
• Avoid deadlock: Functor may call queueInLoop again

when do you need to wake up

• The thread calling queueInLoop is not an IO thread
• queueInLoop is called in the IO thread but pendingFunctor is being called at this time
  • The functor called by doPendingFunctor may call queueInLoop again. Since the functors called in doPendingFunctor are local variables, the re-added functors are invisible in local variables and cannot be executed, so wakeup is required

That is to say, only calling queueInLoop in the IO thread event callback does not need to wake up

EventLoop::quit

If the IO thread calls quit by itself, it means that it is not stuck in the poll at this time, just quit_=true, and it will jump out of the loop when the while judgment is made next time; if other threads call the quit of the IO thread, the IO thread may be stuck at this time In poll, you need to wake up

Channel

Each channel object belongs to only one EventLoop, which encapsulates a file descriptor and the events it is interested in, and passes it to the poller to monitor through the loop. The poller tells the loop to return the event, and the loop calls the corresponding event callback of the channel.

Users generally do not use channels directly but use higher-level encapsulation such as TcpConnection, whose lifetime is managed by its upper layer

bollard

The encapsulation of IO multi-channel is an abstract base class that supports poll and epoll

Poller is an indirect member of EventLoop, only for its owner EventLoop to call in the IO thread, no need to lock, and its lifetime is equal to EventLoop

EpollPoller

EpollPoller::poll

Listen to the channel passed by EventLoop and fill in the events that occurred into the activeChannels passed by EventLoop, and hand it over to EventLoop to complete the event distribution

EpollPoller::fillActiveChannels

Why can't it traverse events while calling Channel::handlerEvent

Simplify Poller's responsibilities so that it is only responsible for IO multiplexing, not event distribution, and can easily replace other IO multiplexing mechanisms in the future

EpollPoller::updateChannel

Modify the state of the channel and hand it over to EpollPoller::update to modify the events monitored in the poller

EpollPoller::update

EventLoopThread

EventLoopThread will start its own thread and run EventLoop::loop in it

Create an EventLoop and notify startLoop that the loop is created, then run the loop

When the thread starts to run, it needs to ensure that the loop is created first, so use the condition variable to achieve synchronization

EventLoopThreadPool

Open the thread pool to create multiple threads

Obtain EventLoop through polling algorithm

Acceptor

Used to receive new connections and notify users through callbacks for use by TcpServer, whose lifetime is managed by it

TcpServer will create Acceptor and give it the port to listen to and the newConnection processing method for new connections. When the Acceptor initializes, it will create listenFd and bind it to the listening port. At the same time, listenFd will be encapsulated into a channel, and the channel can be bound. The callback function handleRead of the read event, handleRead will receive a new connection and execute newConnection

It is not ideal to pass connfd directly to cb. If cb is abnormal, connfd cannot be closed.

• You can first encapsulate connfd as a socket object, and then use the move semantics to move the socket object to the callback function to ensure the safe release of resources. Because the socket object closes the file descriptor when it is destructed

listen

To start monitoring, pass acceptFd to Poller to monitor

TcpServer

Manage the TcpConnection obtained by accep, responsible for its connection establishment, port, data sending and reading

For user use, the lifetime is controlled by the user

accept new connections

Poller will notify EventLoop when it detects that the accept Socket has an event that is readable. EventLoop will call the handleEvent callback method of acceptChannel. After judging that the event is readable, it will call the handleRead corresponding to the channel. In acceptChannel, handleRead receives a new connection and executes TcpServer to pass it to the acceptorChannel The callback function newConnection

Encapsulate the new connection as TcpConnection and set the corresponding callback function (in essence, a channel will be created when TcpConnection is initialized, and TcpConnection will be set to give the callback function), and the fd corresponding to TcpConnection will be handed over to Poller to monitor and execute the new connection provided by the user through connectEsablished incoming callback function

Disconnect

Poller will notify EventLoop when it detects that the fd to close the connection has an event readable. EventLoop will call the handleEvent callback method of its channel. Set handleClose to execute closing connection

handleClose is passed to Channel by TcpConnection

Cancel the event monitoring of the fd owned by the channel in the poller, and execute two callbacks at the same time

connectionCallback_

A callback provided by the user, which handles what the user needs to do when the connection is disconnected (such as telling the user that the connection has been successfully disconnected)

closeCallback_

Provided by TcpServer, bound to TcpServer::removeConnection

Notify TcpServer to remove TcpConnection from its saved connection array, and then unregister channel from EventLoop

Why remove in connectDestroyed instead of remove in handleClose

Because the connection disconnection is not necessarily caused by the client actively disconnecting and calling handleClose, handleClose may not be called

After TcpConnection is removed from connections_, the reference count is decremented by 1. If the user does not hold the shared_ptr pointer of TcpConnection, then calling TcpConnection::connectionDestroyed will not access the destructed object

TcpConnection internally uses bind to bind its shared_ptr when registering the callback function and calls shared_from_this() to obtain it to increase the reference count and extend the life cycle

read data

When TcpConnection is initialized, it will set the event callback that should be executed when the channel it owns has a readable event.

Read data through input buffer

• If the data is read, call the callback function that the user's message has data arrival, and pass the data to the callback function
• If 0 bytes are read, it means that the user wants to disconnect, and the callback function for connection disconnection is executed at this time
• If read < 0, an error occurred and error handling is required

send data

TcpConnection exposes the send interface to other threads. For example, TcpServer can call send to send data to the user, so it is necessary to ensure that the send operation is executed in the IO thread to which it belongs.

sendInLoop

It will try to send data directly first, and will not start writeCallback if it is sent once

If only part of the data is sent, the remaining data will be put into the outputbuffer and start to pay attention to the writable event, and then the remaining data will be sent in handleWrite

handlewrite

When the socket becomes writable, the Channel will call TcpConnection::handleWrite to continue sending the data in the outputBuffer

Once sent, it will stop observing the writeable event to avoid busy loop

When to pay attention to writable events? What if the speed of sending data is higher than the speed of receiving data from the other party, causing data to accumulate in the local memory?

muduo provides two callbacks: high water mark callback and low water mark callback

Low water level callback WriteCompleteCallback: Called when the send buffer is cleared

High Water Mark Callback HighWaterMarkCallback: If the length of the output buffer exceeds the size specified by the user, the callback will be triggered

TcpConnection

Use shared_ptr management, inherit enable_shared_from_this, because the lifetime of TcpConnection is ambiguous, users can also hold TcpConnection

highlights

Limit the maximum number of concurrent connections to the server

Concurrent connections: the number of client connections that a server program can support at the same time

Why limit the number of concurrent connections

• Do not want the service program to be overloaded: the more connections, the more things to be processed and the more resources occupied
• It will lead to the exhaustion of file descriptors, and the program has no available file descriptors to represent new connections, which will cause the program to fail in repeated calls to accept
  • Everything in Linux is represented by a file descriptor. The file descriptors that can be used by each process are limited. If the limit is exceeded, there will be no way to obtain a new file descriptor to represent the corresponding file.
  • When accept returns EMFILE, it means that the file descriptor of the process has reached the upper limit, and there is no way to create a socket file descriptor for the new connection, which means that there is no socket file descriptor to represent the connection, and we cannot close the connection, and the program continues When running back to calling epoll_wait again, it will return immediately because there are new connections waiting to be processed. If there is no available file descriptor, the program will enter the loop

How to avoid file descriptor exhaustion

• Increase the number of file descriptors for the process

  • Treat the symptoms but not the root cause, as long as there are enough clients, the file descriptors will always be exhausted one day

• Prepare a free file descriptor. When the file descriptor is exhausted, close the free file descriptor first to obtain a file descriptor quota for the new connection, and then the acceptor gets the descriptor of the new socket connection, and then immediately closes the file descriptor. Connection, you can gracefully disconnect from the client, and finally reopen an idle file to occupy the pit, in case you encounter this situation again
  

• Set a slightly lower number of file descriptors yourself, and if it exceeds, it will actively close the new connection

  • Add an atomic_int member in Server to indicate the current number of active connections, judge the current number of active connections in onConnection, and kick the connection if it exceeds
  • 

• Handle idle connections in a timely manner

log library

asynchronous log

The synchronization log needs to wait for the log message to be written to the disk before performing subsequent operations, which will cause the program to block on disk writing

Asynchronous logs store log messages in memory, and when they accumulate to a certain amount or reach a certain time interval, the background thread automatically outputs all stored logs to disk

Use double buffering technology: prepare two buffers: A and B, the front end is responsible for filling data into buffer A, and the back end is responsible for writing the data of buffer B into a file; when buffer A is full, exchange A and B, and let the back end write buffer A The data is written to the file, and the front end fills in new log messages to buffer B, and so on

• The front-end thread will not block on the write log
• When the background thread actually writes the log, the log messages have accumulated a lot. At this time, you only need to call the IO function once, and you don’t need to call it every time (batch processing)

log storage

The output form of the log is in the form of a stream. Muduo does not use the iostream library that comes with C++, but implements LogStream by itself, throwing away other unnecessary redundant functions to make the code more efficient

Buffer FixedBuffer

The implementation of the log stream requires a built-in buffer to store the input data

It is a template class, the template parameter is used to specify the size of the buffer data_, and provides a series of buffer operation functions

log stream LogStream

Contains FixedBuffer, which is responsible for storing the log memory to be recorded by the user in the buffer of the log stream. Because it contains multiple data types, the data is first converted into a string type and then stored in the buffer

LogStream implements the form of log stream by overloading <<. No matter which overloaded version it is, it will eventually be append, and append will call append in FixedBuffer

log output

Impl

Classes used as log output should contain instances of LogStream for storing log messages

Impl contains the content of a record: time, file name, line number, log content

Log output class Logger

There is an instance of Impl to save a log message. Loggeer will construct Impl when constructing, that is, it will initialize the file name and line number of impl. Impl will also store the time in the buffer when constructing.

Logger will add the file name and line number of Impl to the buffer when it is destructed, and then call output to flush the log content to the log file

How to ensure that a log is only output once

Some things only need to be done once in multithreading, usually when initializing the application, you can put it in main, but not when writing a library

It can be initialized statically, and it is enough to initialize pthread_once once: in multithreading, although pthread_once will be called in multiple threads, once_init will only be executed once

macro definition logging

Format when used:LOG << "hello dd" <<;

A temporary Logger object will be constructed, and the file name and line number will be passed in, and then the log stream in the Logger will be obtained through stream, and then the log content will be recorded in the buffer by overloading <<. When this statement is executed, it will be Logger is destructed, and the log content can be written to the log file when it is destructed

Each log message will correspond to a temporary object of Logger, which can output the log message in time, and the occupied memory can be released immediately

Backend and log files

AppendFile

Contains a file pointer to an external file to interact with the disk file

Open to the external interface, write to the buffer of the log file, not actually write to the log file on the disk

The contents of the log file buffer are being written to the log file on disk

LogFile

Wrap an AppendFile class instance file_ through unique_ptr, call the append of the LogFile class when the back-end thread writes, and append will call the append of AppendFile through the instance to write the contents of the back-end buffer into the buffer of the log file

Automatic flush buffer

The file stream that interacts with the log file is fully buffered, and only when the file buffer is full or flushed will the contents of the buffer be written to the file

If flush is not called, the data will be written out to the file only when the buffer is full. If the process crashes suddenly, the unwritten data in the buffer will be lost, and if flush is called too many times, no doubt will affect the efficiency

Muduo decides when to flush in two ways

• When the number of append reaches flushEvenyN, flush will be called
• flush every 3 seconds

Roll the log LogFile::rollFile

• File size will be limited
• Find files at different times according to time

rollFile occurs in two situations

• The number of bytes written out to the log file reached the rollover threshold
• arrive a new day

It is not that the first log message of a new day will cause rollFile, but every time the append function is called 1024 times, it will check whether it is a new day

• There may be cases where a new day has arrived but 1024 calls have not been reached, but if 1024 calls have not been reached, it means that there are few log messages, and there is no need to create a new log file
• If you call append every time to judge whether it is a new day, then you need to use functions such as gmtime and gettimeofday to get the time every time, so it may seem that the gain outweighs the loss

Interaction between front end and back end

Muduo implements asynchronous logging through AsyncLogging

Asynchronous logs are divided into front-end and back-end. The front-end is responsible for storing the generated log messages, and the back-end is responsible for writing the log messages to disk.

front end

When the current buffer and the reserve buffer are used, the front end is used to temporarily store the generated log messages. When the current buffer is not enough, the reserve buffer will be used to fill the current buffer.

Buffers is used to store buffers ready to be written to the backend. When the current buffer is full, it will be stored in buffers

The front end only needs to call the append function. If the currentBuffer is enough to put down the current log message, it will call the append function of the buffer to put the message. If it cannot put it down, it will put the currentBuffer into the buffer. At this time, if the preparatory buffer nextBuffer has not been used, Then the ownership of nextBuffer will be transferred to currentBuffer. After the transfer, nextBuffer is NULL, which means it has been used; and if the preparatory buffer itself is NULL, new space will be reallocated for currentBuffer. After the current front-end moves into the buffer, the condition variable will be woken up

rear end

Responsible for interacting with buffers, writing out all the contents of the buffer in buffers to disk, and executing it by opening a thread (backend thread)

The backend thread will loop to check buffers_. If the buffers are empty, the backend thread will sleep for the number of seconds specified by flushInterval (the default is 3 seconds). If there is data in the buffers during this period, the backend thread will be awakened, otherwise it will sleep until the timeout, no matter what kind of wake-up, the currentBuffer will be moved into the buffers, because the back-end thread prepares to output all log messages for each operation, and most of the currentBuffer There are log messages, so even if it is not full, it will be put into buffers, and then newBuffer1 will be used to supplement currentBuffer

buffersToWrite is the back-end buffer queue, responsible for fetching the data in the front-end buffers, and then writing the data out to disk. Therefore, when the currentBuffer is moved into buffersToWrite, the swap function will be called immediately to exchange buffersToWrite and buffers. This part exchanges the memory pointers in the two vectors, which is equivalent to exchanging their respective contents, and buffers become empty. , and all the previous buffers with log messages are all in buffersToWrite

If the preparatory buffer is empty at this time, it means that it has been used, and it will be supplemented with newBuffer2. At this point, the mutex is released. After buffersToWrite obtains the data of buffers, other threads can call append to add log messages normally, because buffers are reset to empty at this time, and buffersToWrite is a local variable, the two do not affect each other

Then write the content in buffersToWrite to the buffer of the log file through the output appedn

Recycle

After writing, the content of the buffer in bufferToWrite has no value, but it can be recycled. Since both newBuffer1 and newBuffer2 may be used and become empty, the elements of buffersToWrite can be used to fill newBuffer1 and 2.

Under normal circumstances, currentBuffer, nextBuffer, newBuffer1, and newBuffer2 do not need to allocate space twice, because buffers and buffersToWrite can form a resource usage ring between them: the front-end moves currentBuffer into buffers and uses nextBuffer to fill currentBuffer, and the back-end thread Move the new currentBuffer into buffers again, then use newBuffer1 and newBuffer2 to fill currentBuffer and nextBuffer, and finally obtain elements from buffersToWrite to fill newBuffer1 and newBuffer2. It can be seen that the resource consumption end is in currentBuffer and nextBuffer, and the resource replenishment end is in newBuffer1 and newBuffer2, if the process is balanced, then these 4 buffers do not need to allocate new space

Finally call flush to write the contents of the log buffer to the log file

After the core dump, find the logs that have not yet been written

The log messages in the asynchronous log are not written out immediately after they are generated, but are first stored in the front buffer currentBuffer or the front buffer queue buffers, and the log messages in the buffer are not flushed to the disk until the right time. If the program core dumps in the middle, how to retrieve the log messages that have not yet been written in the buffer

View the core file, analyze the reason from the core file, see where the program hangs through gdb, and analyze the variables before and after to find out the reason

When the program terminates abnormally or crashes during the running process, the operating system will record the memory status of the program at that time and save it in a file. This behavior is called Core Dump (some Chinese translate it into "core dump"). We can think of core dump as a "memory snapshot", but in fact, in addition to memory information, some key program running states will also be dumped at the same time, such as register information (including program pointers, stack pointers, etc.), memory management information , other processor and operating system status and information. Core dump is very helpful for programmers to diagnose and debug programs, because some program errors are difficult to reproduce, such as pointer exceptions, and core dump files can reproduce the situation when the program goes wrong

timer

Sending signals through alarm and processing SIGALRM signals on the server side to implement a timing mechanism is not very good in multi-threading, and should try to avoid using signals in multi-threading

• The signal interrupts the running thread, and only reentrant functions can be called in signal processing, but not every thread-safe function is reentrant
• If the signal processing needs to modify the global variable, the modified variable must be of type sig_atomic_t, otherwise the interrupted function may not be able to see the changed data of the signal processing immediately after resuming execution, because the compiler may assume that this variable will not Modified elsewhere to optimize memory access

In muduo, the timerfd provided by Linux is used to turn the time into a file descriptor, which becomes readable when the timer expires, which can be easily integrated into IO multiplexing and unified event source

Timer

The encapsulation of timing operations is mainly composed of three elements: the callback function that needs to be executed when the timing arrives, the timing time, and whether it needs to be executed repeatedly

TimerId

User-visible class: only contains a timer pointer and the number of the timer

TimerQueue

timer queue

The data structure requires the ability to efficiently organize unexpired timers, quickly find expired timers based on the current time, and efficiently add and delete timers

• linear table
  • Find O(N)
• binary heap organization priority queue
  • Find O(logN), but functions such as make_heap in the C++ standard library cannot efficiently delete an element in the middle of the heap
• binary search tree map
  • Sort Timer in chronological order, O(logN)
  • You cannot use map<Timestamp,Timer*> directly, and you cannot handle the situation where two timers have the same expiration time
    • use multimap
    • Distinguish key: pair<Timestamp, Timer*> is the key, so that the expiration time is the same in time, but the address must be different

Muduo uses a balanced binary search tree and uses container set management

The functions in the TimerQueue can only be called in the loop to which it belongs, so there is no need to lock them. Each loop has only one TimerQueue to manage the timing time. When the loop is created, the TimerQueue has already created a number, and timerfd has also been added to the poller to monitor

The timing wheel kicks off idle connections

• Each connection saves the time of the last received data, uses a timer to traverse all connections every second, and disconnects the connections that have no activity within the specified time
  • There is only one repeated timer globally, but every timeout needs to check all connections. If the number of connections is relatively large, it will take time
• Set a one-shot timer for each connection, and disconnect when it times out; update the timer every time data is received
  • Multiple one-shot timers are required, and timers will be updated frequently. If the number of connections is relatively large, it will put pressure on TimerQueue
• timing wheel
  • A circular queue composed of n buckets, the first bucket holds connections that will time out after 1 second, and the second bucket holds connections that will time out after 2 seconds. Each connection receives data and puts it in the nth bucket. The timer in every second disconnects the connection in the first bucket and moves the bucket to the end of the queue.
  • You don't need to check all the connections every time, you only need to check the connections in the first bucket (distribute the tasks)

Use weak_ptr to determine whether the connection still exists

Customize a structure Entry to represent the connection, including the weak_ptr of TcpConnection, the destructor of Entry will judge whether the connection exists, and disconnect if it exists

[External link picture transfer failed, the source site may have an anti-leeching mechanism, it is recommended to save the picture and upload it directly (img-kSUEMgPh-1665728911894) (C:\Users\gnezd\AppData\Roaming\Typora\typora-user-images\ image-20220929142800679.png)]

And through boost::any, easy type of data can be stored and Entry can be stored in TcpConnection

Reference counting implements destruction

Using the reference counting method, the EntryPtr is placed in the corresponding grid when the data is received from the connection, and the reference count increases. When the reference count of the Entry decreases to zero, it means that it has not appeared in any grid, and the connection times out. Entry destructor will disconnect

circulae_buffer

It is a fixed-size ring buffer implemented by boost. When the buffer is full, the new data will overwrite the old data.

The elements in the buffer save the Bucket, and the Bucket saves the shared_ptr of all connections within one second, initializes the buffer and fills it with empty space

When there is a connection, the connection will be inserted into the Bucket, and an empty Bucket will be inserted into the buffer every second

Given the characteristics of circular_buffer, the existing connection 1 will automatically roll forward

connection coming

When the connection is established, create an Entry object entry with the corresponding TcpConnection object conn, and put it at the end of the timing wheel queue. In addition, we also need to save the weak reference of entry to the context of conn, because Entry is also used when receiving data, and weak reference does not affect the reference count

new news

When receiving the message, take out the weak reference of Entry from the context of TcpConnection, upgrade it to a strong reference of EntryPtr, and put it at the end of the current timing wheel queue. (When promoted to a strong reference, the reference count +1)

Summarize

Each TcpConnection has a Context variable to save the WeakPtr of Entry. Because of the callback mechanism, each connection needs its associated Entry, and using WeakPtr directly here does not affect its reference count. With context, whenever the server receives a message from the client (onMessage), it can get the weak reference of the Entry associated with the connection, and then promote it to a strong reference and insert it into the circular_buffer, which is equivalent to updating the Entry Connected to the position in the time wheel, the corresponding use_count will increase by 1