People have been asking me about the performance of FastDFS, and I will answer it in one sentence: the FastDFS software itself will not become a performance bottleneck; the performance bottleneck of a FastDFS cluster is usually disk IO. Next, I will share some points that FastDFS has done well in terms of performance (the functional components used are in the basic library libfastcommon).
1. Use a concise and efficient self-developed network framework
Versions v2 to v4 use the popular network framework libevent. In order to remove this dependency and for simplicity and efficiency, v5 uses a self-developed network framework that supports Linux, FreeBSD and SunOS. The network timeout management uses a simple and efficient time round algorithm.
2. Use object pooling and connection pooling
The way the object pool manages memory is similar to the Linux memory management SLAB. Interested students can check the relevant information.
A connection corresponds to a buffer, called a task buffer, which is used to receive and send network data. The task buffer can be read and written by business processing threads to avoid memory copying. The task buffer adopts the method of pre-allocation, and currently uses the incremental pre-allocation method to reduce unnecessary memory usage. The allocation of task buffers is essentially an object pool method.
FastDFS v3 supports the merged storage feature. The trunk server manages the free space of the trunk and uses the object pool (Note: The trunk server in a group is also served by one of the storage servers).
To eliminate the overhead of establishing network connections, the tracker server, storage server and C client SDK all support connection pooling.
3. Cache write method
FastDFS writes logs and binlog files by first writing to the cache and then periodically flushing them to the disk. It is recommended to set the flushing interval to 1 second. libfastcommon implements the function of timed task scheduling (timer) through a thread, and supports dynamic addition and deletion of timed tasks.
4. Other programming skills
1) The current time is a global variable, obtained in the timer thread, and used directly elsewhere;
2) Use gcc's built-in atomic counting function to avoid using locks, such as: __sync_add_and_fetch ;
3) Pre-allocate and initialize buffers with fixed content, such as ASCII 0-filled buffers;
4) Try not to use memset;
5) and so on. . .
The above content has been organized in a hurry, and I believe there are still some missing places. Friends who have read the codes of FastDFS and libfastcommon are welcome to add, feedback and communicate.
This article is shared from the WeChat public account - FastDFS Sharing and Exchange (fastdfs100).
If there is any infringement, please contact [email protected] to delete it.
This article participates in the " OSC Yuanchuang Project ", you are welcome to join and share with us.