Common programming model 04 of muduo library learning-choice of common programming model and inter-process communication mode

Dongyang's study notes

Original link: https://blog.csdn.net/yuyin86/article/details/7086424

1. Commonly used programming model for single-threaded servers

For commonly used single-threaded programming models, see https://blog.csdn.net/qq_22473333/article/details/112910686

Among high-performance network programs, the most widely used model is probably the " non-blocking IO + IO multiplexing " model, that is, the Reactor mode. I know:

l lighttpd, a single-threaded server. (Nginx is estimated to be similar to this, pending investigation)

• libevent/libev

• ACE, Poco C++ libraries (QT pending)

• Java NIO (Selector/SelectableChannel), Apache Mina, Netty (Java)

• POE (Perl)

• Twisted (Python)

On the contrary, boost::asio and Windows I/O Completion Ports implement the Proactor mode, and the application area seems to be narrower. Of course, ACE also implements the Proactor mode, not shown.

Under the "non-blocking IO + IO multiplexing" model, the basic structure of the program is an event loop: (The code is only for illustration, and various situations are not fully considered)

while (!done)

{
    
    
  int timeout_ms = max(1000, getNextTimedCallback());
  int retval = ::poll(fds, nfds, timeout_ms);
  if (retval < 0) {
    
    
    // 处理错误

  } else {
    
    
    // 处理到期的 timers
    if (retval > 0) {
    
    
    // 处理 IO 事件
    }
  }
}

Of course, select(2)/poll(2) has many shortcomings. Linux can be replaced with epoll, and other operating systems also have corresponding high-performance alternatives (search for c10k problem).

ReactorThe advantages of the model are obvious, the programming is simple, and the efficiency is good . Not only network reads and writes can be used, connection establishment (connect/accept) and even DNS resolution can be performed in a non-blocking manner to improve concurrency and throughput. It is a good choice for IO-intensive applications, Lighttpdthat is, its internal fdevent structure is very delicate and worth learning. (Here does not consider the sub-optimal solution of blocking IO.)

Of course, it is not so easy to implement a high-quality Reactor, and I have not used any open source libraries, so it is not recommended here.

2. The threading model of a typical multithreaded server

There are not many documents that I can find in this regard, there are probably so few:

1. Each request creates a thread, using blocking IO operations. Before Java 1.4 introduced NIO, this was the recommended practice for Java network programming. Unfortunately, the scalability is not good.

2. Use thread pool, also use blocking IO operation. Compared with 1, this is a measure to improve performance.

3. Use non-blocking IO + IO multiplexing . That is, the way of Java NIO.

4. Leader/Follower and other advanced modes

By default, I will use the third, non-blocking IO + one loop per thread mode.

http://pod.tst.eu/http://cvs.schmorp.de/libev/ev.pod#THREADS_AND_COROUTINES

One loop per thread

Under this model, each IO thread in the program has one event loop(or called Reactor), which is used to process read, write and timing events (regardless of periodic or single time). The code framework is the same as in Section 2.

The advantages of this approach are:

1. The number of threads is basically fixed and can be set when the program is started, and it will not be created and destroyed frequently.

2. It is easy to allocate load among threads.

event loop It represents the main loop of the thread. Just register the timer or IO channel (TCP connection) to the loop of that thread if you need to let which thread does the work.

• Connections that require real-time performance can use a single thread;
• A connection with a large amount of data can monopolize a thread
• Allocate data processing tasks to other threads;
• Other secondary auxiliary connections can share a thread.

For non-trivial server programs, non-blocking IO + IO multiplexing is generally used . Each connection/acceptor is registered to a Reactor. There are multiple Reactors in the program, and each thread has at most one Reactor.

Multi-threaded programs place higher requirements on Reactor, that is 线程安全. To allow one thread to stuff things into the loop of other threads, the loop must be thread-safe.

Thread Pool

However, the IO thread does not have light computing tasks, use the event loop bit of a waste, I will have a complementary program, which uses blocking queuethe task queue implementation (TaskQueue):

blocking_queue<boost::function<void()> > taskQueue;  // 线程安全的阻塞队列
void worker_thread()

{
    
    
  while (!quit) {
    
    
    boost::function<void()> task = taskQueue.take();  // this blocks
    task();  // 在产品代码中需要考虑异常处理
  }
}

It is particularly easy to implement thread pools in this way:

Start a thread pool with a capacity of N:

int N = num_of_computing_threads;

for (int i = 0; i < N; ++i) {
    
    
  create_thread(&worker_thread);  // 伪代码：启动线程
}

It is also very simple to use:

boost::function<void()> task = boost::bind(&Foo::calc, this);
taskQueue.post(task);

The above dozen or so lines of code implement a simple fixed number of thread pools, which are roughly equivalent to a certain "configuration" of Java 5's ThreadPoolExecutor. Of course, in a real project, these codes should be encapsulated in a class instead of using global objects.

Another thing to note: the lifetime of the Foo object, my other blog "When the destructor encounters multithreading-Thread-safe object callbacks in C++" discusses this issue in detail
http://blog.csdn.net /Solstice/archive/2010/01/22/5238671.aspx

In addition to the task queue, it may also be used blocking_queue<T>consumers to achieve data - producer queue, i.e., T is the data type of the object instead of a function, queue consumer (s) from which to get the data is processed. This is more specific than task queue.

blocking_queue is a powerful tool for multi-threaded programming. Its implementation can refer to (Array|Linked)BlockingQueue in Java 5 util.concurrent. Usually C++ can use deque as the underlying container. The code in Java 5 is highly readable, the basic structure of the code is the same as the textbook (1 mutex, 2 condition variables), and the robustness is much higher. If you don't want to implement it yourself, it's better to use a ready-made library. (I have not used free libraries, where chaos is not recommended, interested students can try Intel Threading Building Blocksin the concurrent_queue.)

induction

To sum up, the multi-threaded server programming model I recommend is: event loop per thread+ thread pool.

• The event loop is used as non-blocking IO and timer.
• The thread pool is used for calculation, which can be a task queue or a consumer-producer queue.

Writing server programs in this way requires a high-quality network library based on the Reactor model to support. I have only used in-house products. I can't compare and recommend common C++ network libraries on the market. Sorry.

The specific parameters used in the program such as the loop and the size of the thread pool need to be set according to the application. The basic principle is "impedance matching" so that both CPU and IO can operate efficiently. I will talk about specific considerations later.

There is no talk about thread exit here, leaving it to the next blog "Multithreaded Programming Anti-pattern" to discuss.

In addition, there may be individual threads that perform special tasks in the loggingprogram. For example , this is basically invisible to the application, but it should be included when allocating resources (CPU and IO) to avoid overestimating the capacity of the system. .

3. Inter-process communication and inter-thread communication

There are countless ways of inter-process communication (IPC) under Linux. UNPv2 lists pipes, FIFOs, POSIX message queues, shared memory, signals, etc., not to mention Sockets. There are also many synchronization primitives, such as mutexes, condition variables, reader-writer locks, record locking, semaphores, and so on.

How to choose? According to my personal experience, the preciousness is not too expensive. Careful selection of three or four things can fully meet my work needs, and I can use them very well, and it is not easy to make mistakes.

Inter-process communication

I prefer Sockets for inter-process communication (mainly referring to TCP, I haven't used UDP, and I don't consider the Unix domain protocol). Its biggest advantage is that it can be cross-hosted and has scalability. Anyway, there are multiple processes. If one machine has insufficient processing power, it is natural to use multiple machines to process it. Distribute the process to multiple machines in the same LAN, and continue to use the program by changing the host:port configuration. On the contrary, none of the other IPCs listed above can cross machines (for example, shared memory is the most efficient, but no matter how it is, it cannot efficiently share the memory of two machines), which limits scalability.

In programming, both TCP sockets and pipe are file descriptors used to send and receive byte streams, and both can read/write/fcntl/select/poll, etc. The difference is that TCP is two-way, pipe is one-way (Linux), two file descriptors must be opened for two-way communication between processes, which is inconvenient; and the process must have a parent-child relationship to use pipe, which restricts the pipe's use. Under the communication model of sending and receiving byte streams, there is no more natural IPC than sockets/TCP. Of course, pipe also has a classic application scenario, which is used to asynchronously wake up select (or equivalent poll/epoll) calls when writing Reactor/Selector (Sun JVM does this in Linux).

The TCP port is exclusively owned by a process, and the operating system will automatically reclaim it (the listening port and the TCP socket of the established connection are both file descriptors, and the operating system will close all file descriptors when the process ends). This shows that even if the program exits unexpectedly, it will not leave garbage to the system. After the program is restarted, it can be recovered relatively easily without restarting the operating system (the risk of using a cross-process mutex). There is another advantage, since the port is exclusive, it can prevent the program from restarting (the latter process can't grab the port, naturally it won't work), causing unexpected results.

Two processes communicate via TCP. If one crashes, the operating system will close the connection, so that the other process will be aware of it almost immediately and can quickly failover. Of course, the heartbeat of the application layer is also indispensable. I will talk about the design of the heartbeat protocol when I talk about the date and time processing of the server in the future.

Compared with other IPCs, a natural benefit of the TCP protocol is 可记录可重现that tcpdump/Wireshark is a good helper for resolving protocol/state disputes between two processes.

In addition, if the network library has a "connection retry" function, we can not require the processes in the system to be started in a specific order, and any process can be restarted separately, which is of great significance to the development of a reliable distributed system.

Using TCP byte stream communication, there will be marshal/unmarshal overhead, which requires us to choose a suitable message format, to be precise, wire format. This will be the subject of my next blog, and I recommend it for now Google Protocol Buffers.

Someone might say that if the two processes are on the same machine, use shared memory, or use TCP. For example, MS SQL Server supports both communication methods at the same time. I asked, is it worthwhile to greatly increase the complexity of the code for such a little performance improvement? TCP is a byte stream protocol, which can only be read sequentially and has a write buffer; shared memory is a message protocol. Process a fills up a block of memory for process b to read, basically a "stop waiting" method. To combine these two methods into one program, you need to build an abstraction layer to encapsulate two IPCs. This will bring opacity, and increase the complexity of the test, and in case one party of the communication crashes, the state reconcile will be more troublesome than sockets. Not taken by me. Besides, are you willing to share machine resources with your program for SQL Server bought for tens of thousands of dollars? The database server in the product is often an independent high-configuration server, and generally does not run other resource-intensive programs at the same time.

TCP itself is a data stream protocol. In addition to directly using it to communicate, you can also build upper-layer communication protocols such as RPC/REST/SOAP on top of it, which is beyond the scope of this article. In addition, in addition to point-to-point communication, application-level broadcast protocols are also very useful, which can easily build a considerable and controllable distributed system.