After 5 months of painstaking research and development, FastCFS v3.0 is finally released. The main improvements of FastCFS 3.0: The core component FastDIR implements the data storage engine through plug-ins, uses binlog + storage engine plug-ins, loads inode data on demand, and supports 10 billion-level massive files with limited memory (such as 64GB) on a single machine.
The data persistence implemented by binlog is relatively simple. When the program restarts, all inode data is loaded into the memory through binlog replay. There are the following two problems in storing massive files in this way:
1. The program takes a long time to start up (loading about 2 million inodes);
2. The memory space requirements are very high (one inode occupies about 300 bytes).
The storage engine plug-in introduced in V3.0 solves the two major problems of data persistence purely through binlog. Follow-up technical articles will introduce the principles and characteristics of the FastDIR storage engine in detail, so stay tuned.
In addition, FastCFS 3.0 fixes the following 3 bugs:
[fdir] increase/decrease parent's nlink on rename operation
[fdir] set dentry->kv_array->count to 0 correctly
[fstore] should init barray->count to 0
The FastCFS open source project is hosted on github and gitee. You are welcome to install and use the latest version of FastCFS. If you have any questions or suggestions, please feel free to communicate.