kafka the consumer when reading data server, you need to send a file server disk to the consumer process through the network, the network sends need to go through several network nodes. As shown below:
The traditional step to read the file and send data to the network as follows:
(1) The operating system reads the data from the disk file to the page buffer in the kernel space;
(2) the application reads data from kernel space user-space buffer ;
(3) application will read and write data back to the kernel space into the socket buffer;
(4) operating system to copy the data from the socket buffer to the network interface, data can be sent over the network at this time.
Under normal circumstances, Kafka messages have multiple subscribers, producer announcement will be repeated consumption of different consumers, in order to optimize this process, Kafka used the "zero-copy technology", as shown below:
"Zero-copy technology" only copy data disk file into the page cache once and then cached send data directly from the page to the network (when sent to different subscribers, you can use the same page caching) to avoid repeat the copy operation.
If there are 10 consumers, the traditional way, the number of data replication is 4 * 10 = 40 times, while the use of "zero-copy technology" requires only 1 + 10 = 11 times, once for copying from disk to cache pages, represents 10 times 10 consumers each read a page cache.
Original: https://www.jianshu.com/p/835ec2d4c170