FastDFS has two main roles: Tracker Server and Storage Server
Tracker Server: Tracker Server, which is mainly responsible for scheduling the communication between storage nodes and clients, plays a role in load balancing in access, and records the running status of storage nodes, which is to connect client and storage nodes the hub.
Storage Server: A storage server that saves files and their meta data (meta data)
Group: A file group, also known as a volume. The files on the servers in the same group are exactly the same. When doing a cluster, there are often multiple servers in a group. After uploading a file to a machine in the same group, FastDFS will instantly synchronize the file to the servers in the same group. On all other machines, it acts as a backup.
meta data: file related attributes, key value pair (Key Value Pair) method, such as: width=1024, height=768. Similar to the meta data of Alibaba Cloud OSS.
Tracker Server: 10.0.31.189
Storage Server: 10.0.31.189
Operating System: CentOS6.8
User: root
Data Storage Directory: /fastdfs
Installation Package:
fastdfs-master-V5.05.zip: FastDFS source code
libfastcommon-master.zip: (Common C function library extracted from FastDFS and FastDHT)
fastdfs-nginx-module-master.zip: storage node http service nginx module
nginx-1.10.0.tar.gz: Nginx installation package
After the download is complete, extract the compressed package to the /usr/local directory
[root@localhost ~]# yum install make cmake gcc gcc-c++
[root@localhost ~]# cd /usr/local
[root@localhost local]# unzip libfastcommon-master.zip
[root@localhost local]# cd libfastcommon-master
[root@localhost libfastcommon-master]# ll
总用量 36
drwxr-xr-x. 2 root root 4096 7月 6 2017 doc
-rw-r--r--. 1 root root 8005 7月 6 2017 HISTORY
-rw-r--r--. 1 root root 566 7月 6 2017 INSTALL
-rw-r--r--. 1 root root 1607 7月 6 2017 libfastcommon.spec
-rwxr-xr-x. 1 root root 3099 7月 6 2017 make.sh
drwxr-xr-x. 2 root root 4096 7月 6 2017 php-fastcommon
-rw-r--r--. 1 root root 2763 7月 6 2017 README
drwxr-xr-x. 3 root root 4096 9月 28 2017 src
####编译、安装
[root@localhost libfastcommon-master]# ./make.sh
[root@localhost libfastcommon-master]# ./make.sh install
[root@localhost ~]# cd /usr/local
[root@localhost local]# unzip fastdfs-master.zip
[root@localhost local]# cd fastdfs-master
[root@localhost fastdfs-master]# ll
总用量 136
drwxr-xr-x. 3 root root 4096 9月 28 2017 client
drwxr-xr-x. 2 root root 4096 9月 28 2017 common
drwxr-xr-x. 2 root root 4096 8月 10 2017 conf
-rw-r--r--. 1 root root 35067 8月 10 2017 COPYING-3_0.txt
-rw-r--r--. 1 root root 3171 8月 10 2017 fastdfs.spec
-rw-r--r--. 1 root root 33207 8月 10 2017 HISTORY
drwxr-xr-x. 2 root root 4096 8月 10 2017 init.d
-rw-r--r--. 1 root root 7755 8月 10 2017 INSTALL
-rwxr-xr-x. 1 root root 5548 8月 10 2017 make.sh
drwxr-xr-x. 2 root root 4096 8月 10 2017 php_client
-rw-r--r--. 1 root root 2380 Aug 10 2017 README.md
-rwxr-xr-x. 1 root root 1768 Aug 10 2017 restart.sh
-rwxr-xr-x. 1 root root 1680 Aug 10 2017 stop.sh
drwxr-xr-x. 4 root root 4096 Sep 28 2017 storage
drwxr-xr-x. 2 root root 4096 Aug 10 2017 test
drwxr-xr-x. 2 root root 4096 Sep 28 2017 tracker
## Compile and install
[root@localhost fastdfs-master]# ./make.sh
[root@localhost fastdfs-master]# ./make.sh install
1. Service Script
/etc/init.d/fdfs_storaged
/etc/init.d/fdfs_trackerd
2. Configuration file (example configuration file):
/etc/fdfs [root@localhost fastdfs-master]# ll /etc/fdfs/ -rw-r--r--. 1 root root 1461 9月 28 2017 client.conf.sample -rw-r--r--. 1 root root 7927 9月28 2017 storage.conf.sample -rw-r--r--. 1 root root 7200 9月 28 2017 tracker.conf.sample
3. Command line tools (under /usr/bin directory)
[root@localhost fastdfs-master]# ll /usr/bin/ | grep fdfs
-rwxr-xr-x. 1 root root 262291 9月 28 2017 fdfs_appender_test
-rwxr-xr-x. 1 root root 261988 9月 28 2017 fdfs_appender_test1
-rwxr-xr-x. 1 root root 252300 9月 28 2017 fdfs_append_file
-rwxr-xr-x. 1 root root 251458 9月 28 2017 fdfs_crc32
-rwxr-xr-x. 1 root root 252407 9月 28 2017 fdfs_delete_file
-rwxr-xr-x. 1 root root 253222 9月 28 2017 fdfs_download_file
-rwxr-xr-x. 1 root root 252948 9月 28 2017 fdfs_file_info
-rwxr-xr-x. 1 root root 265628 9月 28 2017 fdfs_monitor
-rwxr-xr-x. 1 root root 879078 9月 28 2017 fdfs_storaged
-rwxr-xr-x. 1 root root 268651 9月 28 2017 fdfs_test
-rwxr-xr-x. 1 root root 267828 9月 28 2017 fdfs_test1
-rwxr-xr-x. 1 root root 374227 9月 28 2017 fdfs_trackerd
-rwxr-xr-x. 1 root root 253350 9月 28 2017 fdfs_upload_appender
-rwxr-xr-x. 1 root root 256992 9月 28 2017 fdfs_upload_file
[root@localhost fastdfs-master]# cp /etc/fdfs/tracker.conf.sample /etc/fdfs/tracker.conf
[root@localhost fastdfs-master]# vim /etc/fdfs/tracker.conf
###Modify as follows
disabled=false # Enable configuration file
port=22122 # Tracker server port (default 22122)
base_path=/fastdfs/tracker # Storage Root directory for logs and data
[root@localhost fastdfs-master]# mkdir -p /fastdfs/tracker
[root@localhost fastdfs-master]# vi /etc/sysconfig/iptables
Add
-A INPUT -m state --state NEW -m tcp -p tcp --dport 22122 -j ACCEPT to the end of vi /etc/sysconfig/iptables
Save and exit to
restart the firewall:
service iptables restart
[root@localhost fastdfs-master]# /etc/init.d/fdfs_trackerd start
The first startup will generate logs and data in the /fastdfs/tracker directory
Check if FastDFS Tracker Server starts successfully: `
ps -ef | grep fdfs_trackerd
[root@localhost fastdfs-master]# cp /etc/fdfs/storage.conf.sample /etc/fdfs/storage.conf
[root@localhost fastdfs-master]# vi /etc/fdfs/storage.conf
Modify the contents as follows
disabled=false # Enable configuration file
port=23000 # storage service port
base_path=/fastdfs/storage # Data and log file storage root directory
store_path0=/fastdfs/storage # The first storage directory
tracker_server=10.0.31.189: 22122 # tracker server IP and port, here is the ip predetermined above
http.server_port=80 # http access file port, this needs to be consistent with the following nginx listening port
[root@localhost fastdfs-master]# mkdir -p /fastdfs/storage# The base path and store_path0 path configured above here, because the above path may not exist
[root@localhost fastdfs-master]# vi /etc/sysconfig/iptables
Add the following port line:
-A INPUT -m state --state NEW -m tcp -p tcp --dport 23000 -j ACCEPT
Save and exit and
restart the firewall:
[ root@localhost fastdfs-master]# service iptables restart
[root@localhost fastdfs-master]# /etc/init.d/fdfs_storaged start
At the first startup, two directories, logs and data, will be generated in the /fastdfs/storage directory.
Check if FastDFS Tracker Server starts successfully:
ps -ef | grep fdfs_storaged
[root@localhost fastdfs-master]# cp /etc/fdfs/client.conf.sample /etc/fdfs/client.conf
[root@localhost fastdfs-master]# vim /etc/fdfs/client.conf
# Modify the following configuration , others keep the default
base_path=/fastdfs/tracker
tracker_server=10.0.31.189:22122 #Modify to tracker_server address and port
[root@localhost fastdfs-master]# /usr/bin/fdfs_upload_file /etc/fdfs/client.conf /root/logo.png
Return content Return file ID number: /group1/M00/00/00/CgAfvVnMWOiAaxm0AAA5ilsiE-g844.png
If it returns something like that, the upload is successful
FastDFS uses the Tracker server to store files on the Storage server. However, file replication is required between storage servers in the same group, and there is a problem of synchronization delay. The fake Tracker server uploaded the file to 10.0.31.201, and the file ID has been returned to the client after the upload is successful. At this time, the FastDFS storage cluster mechanism will synchronize the file to the same group storage 10.0.31.202. If the file has not been copied, if the client uses this file ID to fetch the file on 10.0.31.126, the file will be inaccessible. mistake. The fastdfs-nginx-module can redirect files to connect to the source server to fetch files, avoiding the file inaccessibility error caused by the client's replication delay.
[root@localhost fastdfs-master]# cd /usr/local
[root@localhost local]# unzip fastdfs-nginx-module-master.zip
Please refer to the Nginx source installation documentation for Nginx installation.
Note: When executing ./configure to configure nginx parameters, you need to compile the fastdfs-nginx-moudle source code as a module:
./configure --prefix=/usr/local/nginx --sbin -path=/usr/bin/nginx --add-module=/usr/local/fastdfs-nginx-module-master/src
[root@localhost local]# cp /usr/local/fastdfs-nginx-module-master/src/mod_fastdfs.conf /etc/fdfs/
[root@localhost local]# vi /etc/fdfs/mod_fastdfs.conf #Modify
the following Configure other configurations by default
connect_timeout=10 # Client access file connection timeout duration (unit: seconds)
base_path=/tmp # Temporary directory
tracker_server=10.0.31.189:22122 # tracker service IP and port
storage_server_port=23000 # storage service port
group_name =group1 # Group name
url_have_group_name=true # Access link prefix plus group name
store_path0=/fastdfs/storage # File storage path
root@localhost local]# cd /usr/local/fastdfs-master/conf
root@localhost local]# cp http.conf mime.types /etc/fdfs/
root@localhost local]# ln -s /fastdfs/storage/data/ /fastdfs/storage/data/M00
Note that the user root below opens the previous comment here and modifies it to root to prevent errors from running later
root@localhost local]# vim /usr/local/nginx/conf/nginx.conf
userroot;
worker_processes 1;
events {
worker_connections 1024;
}
http {
include mime.types;
default_type application/octet-stream;
sendfile on;
keepalive_timeout 65;
server {
##This is consistent with the http.server configured in the previous storage.conf
listen 80;
server_name localhost;
##Configure wildcards to map addresses starting with /group to fastdfs projects, if #only one group1 can write group1s directly
location ~/group([0-9])/M00 {
ngx_fastdfs_module;
}
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root html;
}
}
}
In order to prevent problems, it is best to release port 80 in the firewall. The release method is the same as the other release methods above. After the modification is completed, save and restart the firewall.
root@localhost local]# /usr/bin/nginx
ngx_http_fastdfs_set pid=xxx
The nginx restart command is: /usr/sbin/nginx -s reload
After starting nginx, if ngx_http_fastdfs_set pid=xxx is printed, it means that the fastdfs-nginx-module module is loaded Success, xxx represents the process ID of the module.
The file ID returned after the test upload is: group1/M00/00/00/CgAfvVnMWOiAaxm0AAA5ilsiE-g844.png, and the address accessed by browsing is: http://10.0.31.189/group1/M00/00/00/CgAfvVnMWOiAaxm0AAA5ilsiE-g844. png