FastDFS official introduction
FastDFS is an open source distributed file system similar to Google FS. It is implemented in pure C language and supports UNIX systems such as Linux, FreeBSD, and AIX. It can only access files through proprietary APIs, does not support POSIX interface, and cannot be used by mount. To be precise, Google FS and Google FS like FastDFS, mogileFS, HDFS, TFS are not system-level distributed file systems, but application-level distributed file storage services.
1. FastDFS architecture diagram
Figure 1: FastDFS Architecture Diagram
1) Each tracker server in the Tracker cluster is independent of each other and does not communicate with each other.
2) Each storage group (Volume1, Volume2...) in the storage cluster is independent of each other and does not communicate with each other, that is to say, the data saved between each group is different. However, the storage servers in each group are in a mutual backup relationship, that is to say, the storage servers save the same data.
3) Each storage server will start a separate thread to actively report its status information to each tracker server in the Tracker cluster, including disk usage, file synchronization, and file upload and download statistics.
2. Sequence diagram of file upload and download
Figure 2: File upload sequence diagram
1) Client uploads files to Storage server through Tracker server.
2) The Tracker server returns the IP address and port number of an available Storage server to the Client.
3) The Client directly establishes a connection with one of the Storage servers through the IP address and port returned by the Tracker server and uploads files.
4) After the upload is completed, the Storage server returns a file ID to the Client, and the file upload ends.
Figure 3: File Download Timing Diagram
1) The Client downloads a file on a Storage server in the specified Storage group through the Tracker server (the file name includes the Storage group name).
2) The Tracker server returns the IP address and port number of an available Storage server to the Client.
3) The Client directly establishes a connection with one of the Storage servers through the IP address and port returned by the Tracker server and downloads files.
3. Install and configure FastDFS
1. Operating environment and related software
CentOS 6.5
FastDFS_v5.08.tar.gz
nginx-1.8.1.tar.gz
fastdfs-nginx-module_v1.16.tar.gz
libfastcommon
2. Server planning
| server nickname |
IP address and port |
Remark |
| fastdfs-tracker | 172.18.10.232:22122 | Tracking Server/Scheduling Server |
| fastdfs-storage | 172.18.10.233:23000 | storage server |
3. Install FastDFS
1) First create a tool directory (not required)
[root@fastdfs-storage ~]# mkdir -p /home/oldcat/tools [root@fastdfs-storage ~]# cd /home/oldcat/tools/
2) Download and install the FastDFS dependency package libfastcommon
[root@fastdfs-storage tools]# wget https://codeload.github.com/happyfish100/libfastcommon/zip/master [root@fastdfs-storage tools]# unzip master [root@fastdfs-storage tools]# cd libfastcommon-master/ [root@fastdfs-storage libfastcommon-master]# ls HISTORY INSTALL libfastcommon.spec make.sh php-fastcommon README src [root@fastdfs-storage libfastcommon-master]# ./make.sh [root@fastdfs-storage libfastcommon-master]# ./make.sh install
3）下载并安装FastDFS
[root@fastdfs-tracker tools]# wget http://jaist.dl.sourceforge.net/project/fastdfs/FastDFS%20Server%20Source%20Code/FastDFS%20Server%20with%20PHP%20Extension%20Source%20Code%20V5.08/FastDFS_v5.08.tar.gz [root@fastdfs-tracker tools]# tar xf FastDFS_v5.08.tar.gz [root@fastdfs-tracker tools]# cd FastDFS [root@fastdfs-tracker FastDFS]# ./make.sh && ./make.sh install
采用默认方式安装后的文件及目录：
1）服务脚本：
[root@fastdfs-tracker ~]# ll /etc/init.d/ |grep fdfs -rwxr-xr-x. 1 root root 918 4月 22 22:08 fdfs_storaged -rwxr-xr-x. 1 root root 920 4月 22 22:08 fdfs_trackerd
2）样例配置文件
[root@fastdfs-tracker ~]# ll /etc/fdfs/ 总用量 20 -rw-r--r--. 1 root root 1461 4月 22 22:08 client.conf.sample -rw-r--r--. 1 root root 7927 4月 22 22:08 storage.conf.sample -rw-r--r--. 1 root root 7200 4月 22 22:08 tracker.conf.sample
3）命令行工具
[root@fastdfs-tracker ~]# ll /usr/bin|grep fdfs -rwxr-xr-x. 1 root root 252272 4月 22 22:08 fdfs_appender_test -rwxr-xr-x. 1 root root 252225 4月 22 22:08 fdfs_appender_test1 -rwxr-xr-x. 1 root root 242449 4月 22 22:08 fdfs_append_file -rwxr-xr-x. 1 root root 242013 4月 22 22:08 fdfs_crc32 -rwxr-xr-x. 1 root root 242508 4月 22 22:08 fdfs_delete_file -rwxr-xr-x. 1 root root 243627 4月 22 22:08 fdfs_download_file -rwxr-xr-x. 1 root root 243369 4月 22 22:08 fdfs_file_info -rwxr-xr-x. 1 root root 255657 4月 22 22:08 fdfs_monitor -rwxr-xr-x. 1 root root 863913 4月 22 22:08 fdfs_storaged -rwxr-xr-x. 1 root root 258712 4月 22 22:08 fdfs_test -rwxr-xr-x. 1 root root 257881 4月 22 22:08 fdfs_test1 -rwxr-xr-x. 1 root root 365232 4月 22 22:08 fdfs_trackerd -rwxr-xr-x. 1 root root 243547 4月 22 22:08 fdfs_upload_appender -rwxr-xr-x. 1 root root 244453 4月 22 22:08 fdfs_upload_file
注意：虽然FastDFS区分tracker和storage服务器，但是安装的软件及步骤均相同，只是不同的配置文件而已，因此以上安装适用tracker server和storage server
4、配置跟踪服务器（tracker server）
1）拷贝tracker server和client端样例配置文件并重命名
[root@fastdfs-tracker ~]# cp /etc/fdfs/tracker.conf.sample /etc/fdfs/tracker.conf [root@fastdfs-storage ~]# cp /etc/fdfs/client.conf.sample /etc/fdfs/client.conf
2）编辑tracker server配置文件tracker.conf，需要修改内容如下：
disabled=false（默认为false，表示是否无效） port=22122（默认为22122） base_path=/data/fastdfs/tracker
3）编辑client端的配置文件client.conf，需要修改内容如下
base_path=/data/fastdfs/tracker tracker_server=172.18.10.232:22122
4）创建tracker server数据目录
[root@fastdfs-tracker ~]# mkdir -p /data/fastdfs/tracker
5）测试启动tracker server，启动成功会自动在/data/fastdfs/tracker目录新建data和logs目录
[root@fastdfs-tracker ~]# cd /data/fastdfs/tracker/
[root@fastdfs-tracker tracker]# ls
[root@fastdfs-tracker tracker]# /etc/init.d/fdfs_trackerd start
Starting FastDFS tracker server:
[root@fastdfs-tracker tracker]# ss -lntup|grep 22122
tcp LISTEN 0 128 *:22122 *:* users:(("fdfs_trackerd",3785,5))
[root@fastdfs-tracker tracker]# ls
data logs
6）关闭tracker server
[root@fastdfs-tracker tracker]# /etc/init.d/fdfs_trackerd stop waiting for pid [3785] exit ... pid [3785] exit.
5、配置存储服务器（storage server）
1）拷贝storage server样例配置文件并重命名
[root@fastdfs-storage ~]# cp /etc/fdfs/storage.conf.sample /etc/fdfs/storage.conf
2）编辑storage server配置文件storage.conf，需要修改内容如下：
disabled=false（默认为false，表示是否无效） port=23000（默认为23000） base_path=/data/fastdfs/storage tracker_server=172.18.10.232:22122 store_path0=/data/fastdfs/storage http.server_port=8888（默认为8888，nginx中配置的监听端口那之一致）
3）创建storage server数据目录
[root@fastdfs-storage ~]# mkdir -p /data/fastdfs/storage
4）测试启动storage server，启动成功会自动在/data/fastdfs/tracker目录新建data和logs目录（启动storage server的前提是tracker server必须事先已启动）
[root@fastdfs-storage ~]# cd /data/fastdfs/storage/
[root@fastdfs-storage storage]# ls
[root@fastdfs-storage storage]# /etc/init.d/fdfs_storaged start
Starting FastDFS storage server:
[root@fastdfs-storage storage]# ss -lntup|grep 23000
tcp LISTEN 0 128 *:23000 *:* users:(("fdfs_storaged",3786,5))
[root@fastdfs-storage storage]# ls
data logs
6、文件上传测试
执行如下上传命令：
[root@fastdfs-tracker ~]# /usr/bin/fdfs_upload_file /etc/fdfs/client.conf /home/oldcat/imgs/test.jpg group1/M00/00/00/rBIK6VcaP0aARXXvAAHrUgHEviQ394.jpg 返回文件ID即说明文件已经上传成功
四、存储服务器（storage server）安装并配置nginx
1）安装fastdfs-nginx-module模块
注：FastDFS通过Tracker服务器,将文件放在Storage服务器存储，但是同组存储服务器之间需要进入文件复制，有同步延迟的问题。假设Tracker服务器将文件上传到了192.168.4.125，上传成功后文件ID已经返回给客户端。此时FastDFS存储集群机制会将这个文件同步到同组存储192.168.4.126，在文件还没有复制完成的情况下，客户端如果用这个文件ID在192.168.4.126上取文件,就会出现文件无法访问的错误。而fastdfs-nginx-module可以重定向文件连接到源服务器取文件,避免客户端由于复制延迟导致的文件无法访问错误。
[root@fastdfs-storage tools]# tar xf fastdfs-nginx-module_v1.16.tar.gz [root@fastdfs-storage tools]# cd fastdfs-nginx-module/src/ [root@fastdfs-storage src]# vim config 编辑config文件，执行如下命令进行批量替换并保存退出 :%s+/usr/local/+/usr/+g
2）拷贝fastdfs-nginx-module模块中配置文件到/etc/fdfs目录中并编辑
[root@fastdfs-storage ~]# cp /home/oldcat/tools/fastdfs-nginx-module/src/mod_fastdfs.conf /etc/fdfs/ [root@fastdfs-storage ~]# vim /etc/fdfs/mod_fastdfs.conf 修改内容如下： connect_timeout=10 base_path=/tmp（默认为/tmp） tracker_server=172.18.10.232:22122 storage_server_port=23000（默认配置为23000） url_have_group_name = true store_path0=/data/fastdfs/storage group_name=group1（默认配置为group1）
3）安装nginx依赖库
[root@fastdfs-storage nginx-1.8.1]# yum install -y pcre-devel zlib-devel
4）安装nginx
[root@fastdfs-storage tools]# tar xf nginx-1.8.1.tar.gz [root@fastdfs-storage tools]# cd nginx-1.8.1 [root@fastdfs-storage nginx-1.8.1]# ./configure --prefix=/application/nginx/ --add-module=../fastdfs-nginx-module/src/ [root@fastdfs-storage nginx-1.8.1]# make && make install
5）拷贝FastDFS中的部分配置文件到/etc/fdfs目录中
[root@fastdfs-storage ~]# cp /home/oldcat/tools/FastDFS/conf/http.conf /etc/fdfs/ [root@fastdfs-storage ~]# cp /home/oldcat/tools/FastDFS/conf/mime.types /etc/fdfs/
6）配置nginx，如下所示：
[root@fastdfs-storage ~]# vim /application/nginx/conf/nginx.conf
user root;
worker_processes 1;
events {
worker_connections 1024;
}
http {
include mime.types;
default_type application/octet-stream;
sendfile on;
keepalive_timeout 65;
server {
listen 8888;
server_name localhost;
location ~/group[0-9]/ {
ngx_fastdfs_module;
}
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root html;
}
}
}
illustrate:
a, "user root" is to solve the problem of 404 when downloading operation
b. The port number 8888 corresponds to http.server_port=8888 in /etc/fdfs/storage.conf
c. When there are multiple groups corresponding to storage, the access path has the group name, for example: /group1/M00/00/00/**, corresponding to the nginx configuration:
location ~/group[0-9]/ {
ngx_fastdfs_module;
}
7) Copy the nginx service to the /etc/init.d/ directory and start it
[root@fastdfs-storage ~]# cp /application/nginx/sbin/nginx /etc/init.d/
[root@fastdfs-storage ~]# /etc/init.d/nginx
[root@fastdfs-storage ~]# ss -lntup|grep 8888
tcp LISTEN 0 128 *:8888 *:* users:(("nginx",7308,6),("nginx",7309,6))
8) Access previously uploaded files through a browser