FastCFS has just released version 2.2.0 , and IOPS surpasses Ceph in an all-round way : sequential writes are 6.x times that of Ceph , sequential reads are 2.x times that of Ceph , and random writes are about 2 times that of Ceph . For specific performance test data, see: https://gitee.com/fastdfs100/FastCFS/blob/master/docs/benchmark.md . I believe that many friends will be curious about how FastCFS does it. Next, I will reveal the secret of FastCFS IOPS to beat Ceph .
Instead of discussing the differences in architecture and implementation details, I will directly reveal the key practices for effectively improving IOPS .
FastCFS stores data based on trunk files (the default size is 256MB ), and uses sequential writing in a trunk file. Sequential writing can achieve batches of data to disk, which can effectively solve the problem of write amplification, which determines that the writing performance of FastCFS has obvious advantages compared with the traditional disk storage method, which explains that the random write of FastCFS is about 2 times that of Ceph .
Why FastCFS sequential write can kill Ceph in seconds ? In addition to the sequential write mechanism unique to the FastCFS server , another key factor is that the FastCFS client enables the merge write feature by default. The core performance indicator of a distributed storage system is IOPS , which is subject to two other IO capabilities: disk IO and network IO . Combined writes can greatly improve network IO efficiency and effectively eliminate network IO bottlenecks.
Next, let's talk about read performance. Before v2.2.0 , FastCFS adopted the traditional synchronization model, and the random read performance was slightly worse than Ceph . v2.2.0 adopts the asynchronous model based on Linux libaio , the random read performance has been improved, and the performance is slightly stronger than Ceph . Two advantages of using the libaio asynchronous model: 1. The CPU usage is low, and the processing capacity of one asynchronous read thread is approximately equal to 8~ 16 synchronous read threads; 2. Random read IOPS is better. The tradeoff is that the doorframe is higher, Direct IO needs to be used , and the read buffer , file offset, and number of bytes read all need to be aligned by the device block size (note: the block device size is usually 512 bytes, not 4KB ).
After read uses asynchronous IO ( Direct IO ) , the Linux kernel no longer caches the content of the read file, and needs to implement the read-ahead mechanism by itself. In order to improve the efficiency of sequential reading, FastCFS draws on the read-ahead strategy of the Linux kernel, and implements a simple and efficient pre-reading mechanism on the client side.
For scenarios where data is not shared by multiple nodes (that is, data shared by a single node), the combined write and read-ahead mechanism carefully designed and implemented by FastCFS does not have the problem of data consistency (such as reading dirty data), so these two features default to turned on.
For a more detailed comparison test report, see the document: https://gitee.com/fastdfs100/FastCFS/blob/master/docs/benchmark-20210621.pdf , I hope that friends who have the conditions can also do performance testing.
In the process of testing and use, if you have any questions, please give feedback in time, we are always ready to communicate with you. Friendly reminder, our contact information can be found on the gitee official website: https://gitee.com/fastdfs100/FastCFS .
This article is shared from the WeChat public account - FastDFS Sharing and Exchange (fastdfs100).
If there is any infringement, please contact [email protected] to delete it.
This article participates in the " OSC Yuanchuang Project ", you are welcome to join and share with us.