1. Introduction

FastDFS is an open source distributed file system. It manages files. The functions include file storage, file synchronization, file access (file upload, file download), etc., which solves the problem of large-capacity storage and load balancing. Especially suitable for online services that use files as the carrier, such as photo album websites, video websites, and so on.

FastDFS server has two roles: tracker (tracker) and storage node (storage). The tracker mainly does the scheduling work and plays a load balancing role in access.

The storage node stores files and completes all the functions of file management: storage, synchronization and providing access interfaces. FastDFS also manages the meta data of files. The so-called meta data of a file is the related attributes of the file, expressed in a key value pair, such as width=1024, where the key is width and the value is 1024. File meta data is a list of file attributes, which can contain multiple key-value pairs.

The FastDFS system structure is shown in the figure below:

Both the tracker and the storage node can consist of multiple servers. Servers in trackers and storage nodes can be added or offline at any time without affecting online services. Among them, all servers in the tracker are peer-to-peer, and can be increased or decreased at any time according to the pressure of the servers.

In order to support large capacity, storage nodes (servers) adopt a volume (or group) organization. The storage system is composed of one or more volumes. The files between the volumes are independent of each other. The file capacity of all the volumes is the file capacity of the entire storage system. A volume can consist of one or more storage servers. The files in the storage servers under a volume are the same. Multiple storage servers in the volume play a role in redundant backup and load balancing.

When a server is added to the volume, the system will automatically complete the synchronization of existing files. After the synchronization is completed, the system will automatically switch the newly added server to provide online services.

When storage space is insufficient or about to run out, you can dynamically add volumes. You only need to add one or more servers and configure them as a new volume, thus expanding the capacity of the storage system.

2. Download address

1. Download the basic installation package

https://gitee.com/fastdfs100/fastdfs

Or: https://github.com/happyfish100/fastdfs/releases

2. Depend on the tool download address

https://gitee.com/fastdfs100

Or: https://github.com/happyfish100

3. Basic environment

1. System description

System: CentOS-7-x86_64-Minimal-1708

download link:

http://archive.kernel.org/centos-vault/7.4.1708/isos/x86_64/

2. Install the virtual machine (I chose single node installation)

Virtual machine configuration: 2G memory, 2 core CPU

Refer to the installation process:

https://blog.csdn.net/llwy1428/article/details/89328381

3. Tools: xshell5

4. VMware version: VMware Workstation Pro15

Four, installation and deployment

1. Install basic tools

[root@localhost ~]# yum install -y vim lrzsz gcc gcc-c++ make  pcre pcre-devel zlib zlib-devel openssl openssl-devel lrzsz wget net-tools tree zip unzip git

2. Firewall operation

# 停止防火服务
[root@localhost ~]# systemctl stop firewalld
# 查看防火墙的当前状态
[root@localhost ~]# systemctl status firewalld
# 开机禁止启动防火墙
[root@localhost ~]# systemctl disable firewalld

Or the firewall opens the designated port

https://blog.csdn.net/llwy1428/article/details/99676257

Refer to steps 1, 2, and 3 in part three

3. Create a directory and download installation files

[root@loccalhost ~]# mkdir /opt/fdfs

[root @ loccalhost ~] # cd / opt / fdfs

[root @ loccalhost fdfs] # git clone https://gitee.com/fastdfs100/libfastcommon.git

[root @ loccalhost fdfs] # git clone https://gitee.com/fastdfs100/fastdfs-client-java.git

[root @ loccalhost fdfs] # git clone https://gitee.com/fastdfs100/fastdfs.git

4. Switch directories, compile and install libfastcommon

[root @ loccalhost ~] # cd / opt / fdfs / libfastcommon /

[root@loccalhost libfastcommon]# ./make.sh

[root@loccalhost libfastcommon]# ./make.sh install

View

[root@loccalhost ~]# ls /usr/lib64/libfastcommon.so

Copy the library files under /usr/lib64 to /usr/lib

(Note: At this time, first check whether there is a file libfastcommon.so under /usr/lib, if you already have it, you don’t need to copy it)

[root@loccalhost ~]# cp /usr/lib64/libfastcommon.so /usr/lib

5. Configure trackered to compile and install fastdfs

Switch directory

[root @ loccalhost ~] # cd / opt / fdfs / fastdfs

View catalog

[root@localhost fastdfs]# ll
总用量 132
drwxr-xr-x 3 root root  4096 10月 24 21:20 client
drwxr-xr-x 2 root root   189 10月 24 21:20 common
drwxr-xr-x 2 root root   146 10月 24 21:20 conf
-rw-r--r-- 1 root root 35067 10月 24 21:20 COPYING-3_0.txt
drwxr-xr-x 4 root root    56 10月 24 21:20 docker
-rw-r--r-- 1 root root  3170 10月 24 21:20 fastdfs.spec
-rw-r--r-- 1 root root 37049 10月 24 21:20 HISTORY
drwxr-xr-x 2 root root    27 10月 24 21:20 images
drwxr-xr-x 2 root root    48 10月 24 21:20 init.d
-rw-r--r-- 1 root root  8344 10月 24 21:20 INSTALL
-rwxr-xr-x 1 root root  5517 10月 24 21:20 make.sh
drwxr-xr-x 2 root root   320 10月 24 21:20 php_client
-rw-r--r-- 1 root root  2448 10月 24 21:20 README.md
-rw-r--r-- 1 root root  1700 10月 24 21:20 README_zh.md
-rwxr-xr-x 1 root root  1768 10月 24 21:20 restart.sh
-rwxr-xr-x 1 root root   763 10月 24 21:20 setup.sh
-rwxr-xr-x 1 root root  1680 10月 24 21:20 stop.sh
drwxr-xr-x 4 root root  4096 10月 24 21:20 storage
drwxr-xr-x 2 root root   317 10月 24 21:20 test
drwxr-xr-x 2 root root  4096 10月 24 21:20 tracker

[root@loccalhost fastdfs]# ./make.sh && ./make.sh install

Configuration

Copy the files under /opt/fdfs/fastdfs/conf to /etc/fdfs/

[root @ loccalhost ~] # cp / opt / fdfs / fastdfs / conf / * / etc / fdfs /

Switch directory

[root @ loccalhost ~] # cd / etc / fdfs /

View catalog

[root@localhost fdfs]# ll
总用量 124
-rw-r--r-- 1 root root 23981 10月 24 22:08 anti-steal.jpg
-rw-r--r-- 1 root root  1909 10月 24 22:08 client.conf
-rw-r--r-- 1 root root  1909 10月 24 22:06 client.conf.sample
-rw-r--r-- 1 root root   965 10月 24 22:08 http.conf
-rw-r--r-- 1 root root 31172 10月 24 22:08 mime.types
-rw-r--r-- 1 root root 10246 10月 24 22:08 storage.conf
-rw-r--r-- 1 root root 10246 10月 24 22:06 storage.conf.sample
-rw-r--r-- 1 root root   620 10月 24 22:08 storage_ids.conf
-rw-r--r-- 1 root root   620 10月 24 22:06 storage_ids.conf.sample
-rw-r--r-- 1 root root  9138 10月 24 22:08 tracker.conf
-rw-r--r-- 1 root root  9138 10月 24 22:06 tracker.conf.sample

Create a directory

[root@loccalhost ~]# mkdir /home/fdfs_storage /home/fastdfs

Edit configuration file

[root@loccalhost ~]# vim /etc/fdfs/tracker.conf

disabled = false
port = 22122
base_path = /home/fastdfs
allow_hosts = *
http.server_port = 80

Start tracker

[root@loccalhost ~]# /usr/bin/fdfs_trackerd /etc/fdfs/tracker.conf start

(Other: Restart tracker [root@loccalhost ~]# /usr/bin/fdfs_trackerd /etc/fdfs/tracker.conf restart

停止 tracker [root@loccalhost ~]# /usr/bin/fdfs_trackerd /etc/fdfs/tracker.conf stop )

View startup status

[root@localhost ~]# netstat -lntp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 0.0.0.0:22122           0.0.0.0:*               LISTEN      2617/fdfs_trackerd  
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      875/sshd            
tcp        0      0 127.0.0.1:25            0.0.0.0:*               LISTEN      1305/master         
tcp6       0      0 :::22                   :::*                    LISTEN      875/sshd            
tcp6       0      0 ::1:25

6. Configure storage

[root@localhost ~]# vim /etc/fdfs/storage.conf

group_name = group1
client_bind = true
port = 23000
base_path = /home/fastdfs
store_path0 = /home/fdfs_storage
tracker_server = 192.168.11.11:22122
#tracker_server = *.*.*.*:22122
allow_hosts = *

Note: 192.168.11.11 is the IP of my virtual machine

Start storage

[root@localhost~]# /usr/bin/fdfs_storaged /etc/fdfs/storage.conf start

(Others: restart storage [root@localhost~]# /usr/bin/fdfs_storaged /etc/fdfs/storage.conf restart

停止 storage [root@localhost~]# /usr/bin/fdfs_storaged /etc/fdfs/storage.conf start )

7. Check the startup status

[root@localhost ~]# netstat -lntp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 0.0.0.0:22122           0.0.0.0:*               LISTEN      2617/fdfs_trackerd  
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      875/sshd            
tcp        0      0 0.0.0.0:23000           0.0.0.0:*               LISTEN      2704/fdfs_storaged  
tcp        0      0 127.0.0.1:25            0.0.0.0:*               LISTEN      1305/master         
tcp6       0      0 :::22                   :::*                    LISTEN      875/sshd            
tcp6       0      0 ::1:25

8. View the storage directory

[root@localhost ~]# ls /home/fdfs_storage/data/
00  0A  14  1E  28  32  3C  46  50  5A  64  6E  78  82  8C  96  A0  AA  B4  BE  C8  D2  DC  E6  F0  FA
01  0B  15  1F  29  33  3D  47  51  5B  65  6F  79  83  8D  97  A1  AB  B5  BF  C9  D3  DD  E7  F1  FB
02  0C  16  20  2A  34  3E  48  52  5C  66  70  7A  84  8E  98  A2  AC  B6  C0  CA  D4  DE  E8  F2  FC
03  0D  17  21  2B  35  3F  49  53  5D  67  71  7B  85  8F  99  A3  AD  B7  C1  CB  D5  DF  E9  F3  FD
04  0E  18  22  2C  36  40  4A  54  5E  68  72  7C  86  90  9A  A4  AE  B8  C2  CC  D6  E0  EA  F4  FE
05  0F  19  23  2D  37  41  4B  55  5F  69  73  7D  87  91  9B  A5  AF  B9  C3  CD  D7  E1  EB  F5  FF
06  10  1A  24  2E  38  42  4C  56  60  6A  74  7E  88  92  9C  A6  B0  BA  C4  CE  D8  E2  EC  F6
07  11  1B  25  2F  39  43  4D  57  61  6B  75  7F  89  93  9D  A7  B1  BB  C5  CF  D9  E3  ED  F7
08  12  1C  26  30  3A  44  4E  58  62  6C  76  80  8A  94  9E  A8  B2  BC  C6  D0  DA  E4  EE  F8
09  13  1D  27  31  3B  45  4F  59  63  6D  77  81  8B  95  9F  A9  B3  BD  C7  D1  DB  E5  EF  F9

Fourth, use FastDFS's own tools to test

1. Modify the configuration file client.conf

[root@localhost ~]# vim /etc/fdfs/client.conf

base_path = /home/fastdfs
tracker_server = 192.168.11.11:22122
#tracker_server = *.*.*.*:22122

Note: 192.168.11.11 is the IP of my virtual machine

2. I stored a picture 20201024223330.png under /tmp of the system,

Upload files with commands

[root @ localhost ~] # / usr / bin / fdfs_test /etc/fdfs/client.conf upload /tmp/20201024223330.png

Five, compile and install Nginx

Nginx download link: http://nginx.org/download/

1. Switch directory

[root@localhost opt]# cd /opt

2. Download nginx-1.19.3.tar.gz

[root@localhost opt]# wget http://nginx.org/download/nginx-1.19.3.tar.gz

3. Unzip

[root@localhost opt]# tar zxvf nginx-1.19.3.tar.gz

4. Switch directory and download FastDfs Nginx related configuration

[root@loccalhost ~]# cd /usr/local

[root@loccalhost local]# git clone https://gitee.com/fastdfs100/fastdfs-nginx-module.git

5. Compile and install Nginx

Switch directory

[root@loccalhost ~]# cd /opt/nginx-1.19.3

[root@loccalhost nginx-1.19.3]# ./configure --prefix=/usr/local/nginx --add-module=/usr/local/fastdfs-nginx-module/src/

[root@loccalhost nginx-1.19.3]# make && make install

6. Copy mod_fastdfs.conf in the src directory in the fastdfs-nginx-module installation directory to the /etc/fdfs directory

[root @ loccalhost ~] # cp /usr/local/fastdfs-nginx-module/src/mod_fastdfs.conf / etc / fdfs /

7. Edit the configuration file mod_fastdfs.conf

[root@loccalhost ~]# vim /etc/fdfs/mod_fastdfs.conf

base_path=/home/fastdfs
tracker_server=192.168.11.11:22122
group_name=group1
store_path0=/home/fdfs_storage
url_have_group_name = true

8. Edit the Nginx configuration file nginx.conf

[root@localhost ~]# vim /usr/local/nginx/conf/nginx.conf

worker_processes  1;
pid        /usr/local/nginx/logs/nginx.pid;
events {
    worker_connections  1024;
}
http {
    include       mime.types;
    sendfile        on;    
    keepalive_timeout  65;
    server {
        listen       80;
        server_name  localhost;
        location /group1/M00/ {
		    alias /home/fdfs_storage/data/;
		    ngx_fastdfs_module;
	    }
    }
}

9. Start Nginx service

[root@localhost ~]# /usr/local/nginx/sbin/nginx

(Other Nginx operation reference

https://blog.csdn.net/llwy1428/article/details/85063944

https://blog.csdn.net/llwy1428/article/details/101013214 ）

10. View the startup status of system services