1 Introduction

Recent use to fastDFS, so their study a little, a lot of problems encountered in the process of building FastDFS, in order to be able to help build the future FastDFS students, detours, to share with you. Open source FastDFS Taobao senior author Yuqing architecture, this excellent lightweight and distributed file system did not take long, flew on fire. Due to the larger space, a total of four this post, the first chapter introduces FastDFS, download packages, ready to build. The second building will be mentioned in detail the process FastDFS under CentOS. Title III, will be referred to integrate nginx details. Part IV mainly about how to use the FastDFS fastdfs-client-java connection with an example.

2. scenarios

FastDFS is tailor-made for the Internet applications of a distributed file storage system, ideally suited for storing user pictures, videos, documents, and other documents. For Internet applications, and other distributed file systems, compared advantage is obvious. There are several companies that make network disk. Where the storage group number stored in the largest one, the cluster has 400, more than 800 storage server, storage capacity up to 6PB, file number over 100 million, Group sustained growth.
The following is a list of users using FastDFS:
UC ( http://www.uc.cn/ , storage capacity of more than 10TB)
Alipay ( http://www.alipay.com/ )
Jingdong Mall ( HTTP: //www.360buy. COM / )
Tao Tao search ( http://www.taotaosou.com/ )
fetion ( http://feixin.1008.cn/ )
go to the market ( http://www.ganji.com/ )
wash rice ( HTTP : //www.61.com/ )
Thunder ( http://www.xunlei.com/ )
ant cellular ( http://www.mafengwo.cn/ )
Yaya network (http://www.iyaya.com/ )
Rainbow Network ( http://3g.ahong.com )
5173 ( http://www.5173.com/ )
China original network ( http://www.yuanchuang.com / )
China Normal University in Beijing Education cloud platform ( http://www.hsjdy.com.cn/ )
as the Friends network ( http://www.cuctv.com/ )
search Road network ( http://www.sodao.com/ )
58 city ( http://www.58.com/ )
business Alliance network ( http://www.biz72.com/ )
Youth net ( http://www.youth.cn/ )
Bin Li network ( HTTP : //www.binliy.com/ )
flying cloud view video ( http://www.freeovp.com/ )
dream Bazaar ( http://www.moonbasa.com/)
Activities help ( http://www.eventsboom.com )
51CTO ( http://www.51cto.com/ )
SouFun ( http://www.soufun.com/ )

3. Details

FastDFS is an open source distributed file system, she manages files, features include: file storage, file synchronization, file access (file upload, file download), etc., to solve the high-capacity storage and load balancing issues. Particularly suitable for the carrier to file online services, such as photo album sites, video sites, and more.

FastDFS server has two roles: Tracker (Tracker) , and memory storage node (Storage) . Tracker main job scheduling work, since load balancing role in access.

File storage node, all complete file management functions: storage, synchronization, and provide access to interfaces, while FastDFS meta data file management. related meta data is called file attribute of the file to the key-value pair (key value pair) represented manner, such as: width = 1024, which is the key width, value 1024. File meta data is a property list file can contain multiple pairs.

FastDFS system configuration as shown below:

Tracking and storage nodes may be formed by a single multiple servers. Tracker server and storage node can be increased at any time without affecting the online or offline service. Wherein the tracker servers are all equal, the pressure may be increased or decreased according to the server at any time.

In order to support a large capacity, the storage node (server) using the sub-volume (or packet) is organized. Storage system consists of one or more volumes that, between the volume with the volume file are independent, the file capacity of all volumes is the file accumulation capacity of the overall storage system. A volume may be formed from one or more storage servers, the storage server under a file volume is the same, the volume of the multiple storage servers played redundancy and load balancing effect.

Increase in the volume server, the existing file synchronization is completed automatically by the system, after synchronization is complete, the system automatically switches to the new server online service.

When memory is low or about to run out, you can dynamically add volume. Only need to add one or more servers, and configure them for a new roll, so that the expansion of the capacity of the storage system.
FastDFS document identification is divided into two parts: the volume name and file name are indispensable.

3.1 file upload interactive process:

client to upload inquiry tracker storage, no additional parameters;
Returns a tracker storage available;
direct client communication and storage complete the file upload

3.2 Download file interaction:

client inquiries storage tracker download a file, the file parameter identification (the volume name and file name);
Returns a tracker storage available;
direct client communication and storage complete the file download.

4.FastDFS build tool download

Author GitHub address: https://github.com/happyfish100
all the tools to build, can be downloaded on it to. I built is the latest version Version 5.11 2017-05-26. There is a release of 5.10, if you do not have to download the most recent, to the time when the integration of nginx may encounter:

local/fastdfs-nginx-module/src/common.c:1245: 错误：‘FDFSHTTPParams’没有名为‘support_multi_range’的成员
make[1]: *** [objs/addon/src/ngx_http_fastdfs_module.o] 错误 1
make[1]: Leaving directory `/usr/local/nginx-1.10.1'

The reason this error is encountered in fastdfs-nginx-module of HISTORY you can go to:

Integration time, fastdfs-nginx-module in support_multi_range found in Version 5.10 in.

Version 5.11对应的fastdfs-nginx-module的Version 1.20
Version 5.10对应的fastdfs-nginx-module的Version 1.19

之所以在安装前写了这么一段话，是因为这个很重要，版本不对应会给接下来的安装带来各种问题。

把源码下载下来3个zip包，再去下个nginx：

点这里看第二篇

参考：https://www.cnblogs.com/guigujun/p/7804670.html

1.绪论

最近要用到fastDFS，所以自己研究了一下，在搭建FastDFS的过程中遇到过很多的问题，为了能帮忙到以后搭建FastDFS的同学，少走弯路，与大家分享一下。FastDFS的作者淘宝资深架构余庆，这个优秀的轻量及的分布式文件系统的开源没多久，立马就火了。由于篇幅较大，本博文共四篇，第一篇主要介绍FastDFS，下载相关软件包，为搭建做好准备。第二篇会讲到在CentOS下搭建FastDFS的详细过程。第三篇，会讲到整合nginx详细内容。第四篇主要讲FastDFS在怎么用fastdfs-client-java连接，并结合实例。

2.应用场景

FastDFS是为互联网应用量身定做的一套分布式文件存储系统，非常适合用来存储用户图片、视频、文档等文件。对于互联网应用，和其他分布式文件系统相比，优势非常明显。其中有好几家是做网盘的公司。其中存储量最大的一家，集群中存储group数有400个，存储服务器超过800台，存储容量达到6PB，文件数超过1亿，Group持续增长中。
以下是使用FastDFS的用户列表：
UC （http://www.uc.cn/，存储容量超过10TB）
支付宝（http://www.alipay.com/）
京东商城（http://www.360buy.com/）
淘淘搜（http://www.taotaosou.com/）
飞信（http://feixin.1008.cn/）
赶集网（http://www.ganji.com/）
淘米网（http://www.61.com/）
迅雷（http://www.xunlei.com/）
蚂蜂窝（http://www.mafengwo.cn/）
丫丫网（http://www.iyaya.com/）
虹网（http://3g.ahong.com）
5173（http://www.5173.com/）
华夏原创网（http://www.yuanchuang.com/）
华师京城教育云平台（http://www.hsjdy.com.cn/）
视友网（http://www.cuctv.com/）
搜道网（http://www.sodao.com/）
58同城（http://www.58.com/）
商务联盟网（http://www.biz72.com/）
中青网（http://www.youth.cn/）
缤丽网（http://www.binliy.com/）
飞视云视频（http://www.freeovp.com/）
梦芭莎（http://www.moonbasa.com/）
活动帮（http://www.eventsboom.com）
51CTO（http://www.51cto.com/）
搜房网（http://www.soufun.com/）

3.详细介绍

FastDFS是一个开源的分布式文件系统，她对文件进行管理，功能包括：文件存储、文件同步、文件访问（文件上传、文件下载）等，解决了大容量存储和负载均衡的问题。特别适合以文件为载体的在线服务，如相册网站、视频网站等等。

FastDFS服务端有两个角色：跟踪器（tracker）和存储节点（storage）。跟踪器主要做调度工作，在访问上起负载均衡的作用。

存储节点存储文件，完成文件管理的所有功能：存储、同步和提供存取接口，FastDFS同时对文件的meta data进行管理。所谓文件的meta data就是文件的相关属性，以键值对（key value pair）方式表示，如：width=1024，其中的key为width，value为1024。文件meta data是文件属性列表，可以包含多个键值对。

FastDFS系统结构如下图所示：

跟踪器和存储节点都可以由一台多台服务器构成。跟踪器和存储节点中的服务器均可以随时增加或下线而不会影响线上服务。其中跟踪器中的所有服务器都是对等的，可以根据服务器的压力情况随时增加或减少。

为了支持大容量，存储节点（服务器）采用了分卷（或分组）的组织方式。存储系统由一个或多个卷组成，卷与卷之间的文件是相互独立的，所有卷的文件容量累加就是整个存储系统中的文件容量。一个卷可以由一台或多台存储服务器组成，一个卷下的存储服务器中的文件都是相同的，卷中的多台存储服务器起到了冗余备份和负载均衡的作用。

在卷中增加服务器时，同步已有的文件由系统自动完成，同步完成后，系统自动将新增服务器切换到线上提供服务。

当存储空间不足或即将耗尽时，可以动态添加卷。只需要增加一台或多台服务器，并将它们配置为一个新的卷，这样就扩大了存储系统的容量。
FastDFS中的文件标识分为两个部分：卷名和文件名，者缺一不可。

3.1 上传文件交互过程：

client询问tracker上传到的storage，不需要附加参数；
tracker返回一台可用的storage；
client直接和storage通讯完成文件上传

3.2 下载文件交互过程：

client询问tracker下载文件的storage，参数为文件标识（卷名和文件名）；
tracker返回一台可用的storage；
client直接和storage通讯完成文件下载。

4.FastDFS搭建工具下载

作者的GitHub地址：https://github.com/happyfish100
这次搭建的所有工具，都可以在上面下载到。我搭建的是目前最新版本Version 5.11 2017-05-26。作者还有一个5.10的发行版本，你如果没有下载最近的，到时候整合nginx的时候可能会遇到：

local/fastdfs-nginx-module/src/common.c:1245: 错误：‘FDFSHTTPParams’没有名为‘support_multi_range’的成员
make[1]: *** [objs/addon/src/ngx_http_fastdfs_module.o] 错误 1
make[1]: Leaving directory `/usr/local/nginx-1.10.1'

遇到这个错误的原因是，在fastdfs-nginx-module的HISTORY中你可以到：

整合的时候，fastdfs-nginx-module中的support_multi_range在Version 5.10中找不到。