FastCFS actual combat half a year growth record

    In May of this year, Company A intends to replace MooseFS with FastCFS, mainly due to two considerations: 1. Localized replacement; 2. FastCFS has no single-point problems, and the author provides support, which makes it easier and more assured to use. Company A mainly uses MooseFS as the back-end storage to run the qemu virtual machine, and uses the qcow2 format image file.

    Company A conducted strict and meticulous tests on FastCFS, which contributed to the significant improvement of FastCFS in terms of reliability, stability and performance. In terms of reliability and stability improvements, FastCFS has released the following versions:

    FastCFS V3.4 was released on June 15, introducing election nodes, supporting active-active mutual backup to prevent split-brain, and fully supporting POSIX file locks;

    FastCFS V3.5 was released on July 26, which supports majority data replication and strictly guarantees data consistency in extreme cases;

    FastCFS V3.5.1 was released on August 16, more than half of the written data is self-adaptive, and the use of two copies can also ensure data consistency to the greatest extent.

    Compared with similar storage software, FastCFS has obvious advantages in performance, and we have always been proud of it: FastCFS V2.2, released in June 2021, has significantly better performance than Ceph (using SSD disk test): sequential writing is Ceph 6.x times, sequential reads are 2.x times that of Ceph, and random writes are about 2 times that of Ceph.

    However, Company A conducted a performance comparison test between FastCFS and MooseFS in August under the condition of using two SATA disks, which subverted our cognition. The performance of FastCFS cannot run up, making the performance of MooseFS completely crush FastCFS, and the sequential write performance we are proud of is also not good. This test result inspired our fighting spirit. After 3 weeks, FastCFS V3.6 was released on September 8. The file read and write performance has been greatly improved, the write performance has obviously surpassed MooseFS, and the read performance is basically the same as MooseFS. The performance comparison test results of V3.5 before improvement and V3.6 after improvement and MooseFS are as follows (according to the test data provided by Company A): 
105865aa2a8cd01707481d987f7bbb7c.png

    In the case of using SATA disk, why can the performance of V3.6 achieve such a huge improvement? The following will reveal the main work of V3.6 performance optimization for you:

  1. fstore server optimization

    1) File writing supports asynchronous writing to disk to give full play to disk writing capabilities; in order to improve file writing performance, fstore adds a configuration parameter fsync_every_n_writes, the default value is 0, which means that fsync is not explicitly called. For mechanical hard disks (such as SATA disks), not calling fsync can significantly improve the writing performance.

    2) By default, system cache is used for file reading, which can significantly improve reading performance for SATA disks or SAS disks.

  2. fuse client optimization

    1) Support the fuse option writeback_cache, tell the Linux kernel whether to enable coalescing write. Enabling this option will drastically improve the performance of continuously writing small blocks of data (such as writing 4KB at a time).

    2) The configuration item kernel_cache is added to fuse.conf, indicating whether to use the file cache of the Linux kernel. Enabling this option is equivalent to enabling file caching on the fuse client, which will greatly improve file reading performance in some cases.

   Through the above optimizations, the performance of V3.6 has been further expanded compared with Ceph. Interested friends are welcome to conduct comparative tests.

    The current version of FastCFS is V3.6.2. The fdir storage engine plug-in that supports tens of billions of files can be used in the production environment. You are welcome to test and use it.

Guess you like

Origin blog.csdn.net/weixin_38130500/article/details/127274772