Introduction to Muduo Network Library
Muduo is a modern C++ network library based on Reactor mode, author Chen Shuo. It uses a non-blocking IO model, based on event-driven and callbacks, natively supports multi-core and multi-threading, and is suitable for writing Linux server-side multi-threaded network applications.
The core code of muduo network library is only a few thousand lines. In the advanced stage of network programming technology learning, muduo is an open source library that is very worth learning. At present, I have just started to learn the source code of this network library, hoping to record this learning process.
The key structure of Reactor
The core of Reacor is the event distribution mechanism, that is, the io event obtained from the IO reuse model (poll/epoll) is distributed to the event processing function of each file descriptor. In order to implement this mechanism, we need to write Channel class and Poller class. The two classes are explained below.
1 Channel class
According to the author's introduction, the function of the Channel class is similar to the combination of SelectableChannel and Selection-Key of Java NIO. Channel serves the file descriptor fd, it does not occupy fd, and it will not be closed during destruction. Channel will only belong to one EventLoop and only one IO thread. Finally, it should be noted that users generally do not use Channel directly, but use a higher level of encapsulation. Let’s take a look at Channel’s code
class Channel : boost::noncopyable
{
public:
typedef boost::function<void()> EventCallback;
typedef boost::function<void(Timestamp)> ReadEventCallback;
Channel(EventLoop* loop,int fd);
~Channel();
void handleEvent(Timestamp receiveTime);
void setReadCallback(const ReadEventCallback& cb)
{ readCallback_ = cb;}
void setWriteCallback(const EventCallback& cb)
{ writeCallback_ = cb;}
void setCloseCallback(const EventCallback& cb)
{ closeCallback_ = cb;}
void setErrorCallback(const EventCallback& cb)
{ errorCallback_ = cb;}
void tie(const boost::shared_ptr<void>&);
int fd() const {return fd_;}
int events() const {return events_;}
void set_revents(int revt) {revents_ = revt;}
bool isNoneEvent() const {return events_ == kNoneEvent;}
void enableReading() {events_ |= kReadEvent; update();}
void enableWriting() {events_ |= kWriteEvent; update();}
void disableReading() {events_ &= ~kReadEvent; update();}
void disableWriting() {events_ &= ~kWriteEvent; update();}
void disableAll() {events_ &= ~kNoneEvent; update();}
bool isWriting() const {return events_ & kWriteEvent;}
//for Poller
int index() {return index_;}
void set_index(int idx) {index_ = idx;}
//for debug
string reventsToString() const;
void doNotLogHup() {logHup_ = false;}
EventLoop* ownerLoop() {return loop_;}
void remove();
private:
void update();
void handleEventWithGuard(Timestamp receiveTime);
static const int kNoneEvent;
static const int kReadEvent;
static const int kWriteEvent;
EventLoop* loop_;
const int fd_;
int events_;
int revents_;
int index_;
bool logHup_;
boost::weak_ptr<void> tie_;
bool tied_;
bool eventHandling_;
ReadEventCallback readCallback_;
EventCallback writeCallback_;
EventCallback closeCallback_;
EventCallback errorCallback_;
};
1.1 Data members
Among the key data members of the channel, events_ are the IO events it cares about, and there are user settings. revents_ is the current activity event, which is set by EventLoop/Poller; and index_ is the subscript of the struct pollfd array corresponding to the fd served by the channel. The update function will call the update function of EventLoop, and then call the update function of Poller.
1.2 handleEvent function
This function is called by EventLoop::loop, and its function is to call different user callbacks according to the value of revents_.
2 Poller class
Poller is an abstract base class. This class is an encapsulation of the IO reuse model. PollPoller (supporting Poll) or EpollPoller (supporting Epoll) will be implemented later to inherit it. Take a look at Poller's code.
class Poller : boost::noncopyable
{
public:
typedef std::vector<Channel*> ChannelList;
Poller(EventLoop* loop);
virtual ~Poller();
virtual Timestamp poll(int timeoutMs,ChannelList* activeChannels) = 0;
//changes the intrested I/O event
virtual void updateChannel(Channel* channel) = 0;
//remove the channel
virtual void removeChannel(Channel* channel) = 0;
static Poller* newDefaultPoller(EventLoop* loop);
void assertInLoopThread()
{
ownerLoop_ -> assertInLoopThread();
}
private:
EventLoop* ownerLoop_; //the eventloop poller belongs to
};
Poller's virtual function poll completes Poller's core function: get the currently active IO events, and then add them to activeChannels. updateChannel and removeChannel are respectively used to add or remove the fd corresponding to a Channel from the struct pollfd array.
2.1 PollPoller class
The PollPoller class inherits Poller and uses the poll function to implement IO reuse. We are currently using PollPoller, and we will add EpollPoller in the future. The fillActiveChannels of the PollPoller class is to find the fd with the event and fill in the activeChannels. ChannelMap is a map of file descriptors and Channel pointers.
class PollPoller : public Poller
{
public:
PollPoller(EventLoop* loop);
virtual ~PollPoller();
virtual Timestamp poll(int timeoutMs,ChannelList* activeChannels);
virtual void updateChannel(Channel* channel);
virtual void removeChannel(Channel* channel);
private:
void fillActiveChannels(int numEvents,ChannelList* activeChannels) const;
typedef std::vector<struct pollfd> PollFdList;
typedef std::map<int,Channel*> ChannelMap; //key:fd,value:pointer of Channel
PollFdList pollfds_;
ChannelMap channels_;
};
2.1.1updateChannel function
The role of this function maintains and updates the pollfd_ array. It can add file descriptors to the array, or temporarily ignore the event of a file descriptor or re-focus on it. Note that when ignoring file descriptors, use the method to reverse its file descriptor minus one, which can help remove. (The minus 1 is because the file descriptor may be 0).
void PollPoller::updateChannel(Channel* channel)
{
Poller::assertInLoopThread();
LOG_TRACE << "fd = " << channel -> fd() << " events = " << channel -> events();
if(channel -> index() < 0)
{
//new channel,add to pollfds
assert(channels_.find(channel ->fd()) == channels_.end());
struct pollfd pfd;
pfd.fd = channel -> fd();
pfd.events = static_cast<short>(channel -> events());
pfd.revents = 0;
pollfds_.push_back(pfd);
int idx = static_cast<int>(pollfds_.size()) - 1;
channel -> set_index(idx);
channels_[pfd.fd] = channel;
}
else
{
//update exsiting one
assert(channels_.find(channel -> fd()) != channels_.end());
assert(channels_[channel -> fd()] == channel);
int idx = channel -> index();
assert(0 <= idx && idx < static_cast<int>(pollfds_.size()));
struct pollfd& pfd = pollfds_[idx];
assert(pfd.fd == channel -> fd() || pfd.fd == -channel -> fd() - 1);
pfd.events = static_cast<short>(channel -> events());
pfd.revents = 0;
if(channel -> isNoneEvent())
{
pfd.fd = -channel->fd() - 1;
}
}
}
2.1.2removeChannel function
The complexity of the removeChannel function is O(1), and the method it uses is to exchange the Channel to be deleted with the last channel in the array, and then pop_back.
void PollPoller::removeChannel(Channel* channel)
{
Poller::assertInLoopThread();
LOG_TRACE << "fd = " << channel -> fd();
assert(channels_.find(channel -> fd()) != channels_.end());
assert(channels_[channel->fd()] == channel);
assert(channel -> isNoneEvent());
int idx = channel -> index();
assert(0 <= idx && idx < static_cast<int>(pollfds_.size()));
const struct pollfd& pfd = pollfds_[idx]; (void) pfd;
assert(pfd.fd == -channel -> fd() - 1 && pfd.events == channel -> events());
size_t n = channels_.erase(channel -> fd());
assert(n == 1);(void) n;
if(implicit_cast<size_t>(idx) == pollfds_.size() - 1)
{
pollfds_.pop_back();
}
else
{
int channelAtEnd = pollfds_.back().fd;
iter_swap(pollfds_.begin() + idx,pollfds_.end() - 1);
if(channelAtEnd < 0)
{
channelAtEnd = -channelAtEnd - 1;
}
channels_[channelAtEnd] -> set_index(idx);
pollfds_.pop_back();
}
}
summary
After we complete the above two classes, the core content of Reactor is constructed. In the loop function of EventLoop, we can use Poller to get active Channels, and then we can process these active Channels separately, and so on. The sequence diagram is as follows:
