1. Cluster planning
| name of server | IP address |
|---|---|
| fastdsf-tracker1 | 192.168.0.250 |
| fastdsf-tracker2 | 192.168.0.251 |
| fastdfs-storage1-group1 | 192.168.0.252 |
| fastdfs-storage2-group1 | 192.168.0.253 |
| fastdfs-storage3-group2 | 192.168.0.254 |
| fastdfs-storage4-group2 | 192.168.0.249 |
2. Install the required resource package
| Installation package | effect |
|---|---|
| FastDFS_v5.08.tar.gz | FastDFS source code |
| libfastcommon-master.zip | Common C library extracted from FastDFS and FastDHT |
| fastdfs-nginx-module-master.zip | storage node http service nginx module |
| nginx-1.10.0.tar.gz | Nginx installation package |
| ngx_cache_purge-2.3.tar.gz | Image cache clearing Nginx module (used in cluster environment) |
Installation package download link: Click here to download .
3. Cluster deployment
1. Create a new "software" directory under the "/usr/local/" directory to store our installation files. The advantage of this is that it is easy to manage.
cd /usr/local
mkdir software
2. Upload the downloaded installation package to the software directory.
3. Install gcc, and install the gcc environment on all 6 virtual machines.
yum install make cmake gcc gcc-c++
4. There is no zip and unzip command for the minimal installation, so it needs to be installed. The installation command:
yum install zip unzip
5. Unzip libfastcommon-master.zip. Use the command:
unzip libfastcommon-master.zip -d /usr/local/fast/
6. Install vim. The reason why vim is installed is because installing vim will automatically install perl for us, otherwise we will report an error when we use ./make.sh to compile. The command used to install vim:
yum install vim-enhanced
7. To compile libfastcommon-master, we go to the /usr/local/fast/libfastcommon-master directory and use the command: ./make.sh to compile.
8. Then execute the ./make.sh install command
9. Create a soft link. We can see from the above figure that fastcommon is installed under /usr/lib64 after executing the ./make.sh install command. This directory is not commonly used by us. directory, if the soft link is not practical, there will be more things installed in the future, and their installation directory will be difficult for us to find, which is not convenient for management and operation. In order to see the directory we installed in the directory we are familiar with, we use soft links to fulfill.
The command to create the first soft link:
ln -s /usr/lib64/libfastcommon.so /usr/local/lib/libfastcommon.so
Then create three soft links, the commands are as follows:
[root@itcast05 lib]# ln -s /usr/lib64/libfastcommon.so /usr/lib/libfastcommon.so
[root@itcast05 lib]# ln -s /usr/lib64/libfdfsclient.so /usr/local/lib/libfdfsclient.so
[root@itcast05 lib]# ln -s /usr/lib64/libfdfsclient.so /usr/lib/libfdfsclient.so
10. To install FastDFS, we first go to the /usr/local/softWare/ directory, and then use the command:
tar -zxvf FastDFS_v5.08.tar.gz -C /usr/local/fast/
Unzip it, as shown in the figure below.
After decompression, we enter the /usr/local/fast/fastDFS/ directory and execute in sequence
./make.sh
./make.sh install
command to install.
11. Because the bin directory set by the FastDFS service script is under /usr/local/bin/, but actually we installed it under /usr/bin/. So we need to modify the path in the FastDFS configuration file, that is, we need to modify two configuration files, enter the command:
vim /etc/init.d/fdfs_storaged
Enter a ":" followed by a global replace command:
%s+/usr/local/bin+/usr/bin
And press Enter to complete the replacement. After the replacement, save and exit the file, and then open it to see if /usr/local/bin has been replaced with /usr/bin.
In the same steps, enter the second command:
vim /etc/init.d/fdfs_trackerd
To make a global replacement, replace the command:
%s+/usr/local/bin+/usr/bin
12. What we have done before is the configuration of the public part. Next, let’s configure tracker1 (192.168.0.250) and tracker2 (192.168.0.251). We first go to the /etc/fdfs/ directory and use
cp tracker.conf.sample tracker.conf
Make a copy of tracker.conf.sample and rename it tracker.conf.
Edit the tracker.conf file, there are two places that need to be modified, the first one: base_path, change the default path to /fastdfs/tracker. The second place: store_lookup, the value is 2 by default (that is, the load balancing strategy), and now change it to 0 (that is, the polling strategy, which is convenient for us to test later, of course, it must be changed back to 2 in the end. If the value is 1, it means that the upload and download operation should always be performed to a certain group, then "store_group=group2" in the figure below will work, if the value is 0 or 2, then "store_group=group2" will not work).
Since the path /fastdfs/tracker we configured for base_path does not currently exist, we need to create this directory on 192.168.156.5 and 192.168.156.6 and create the command:
mkdir -p /fastdfs/tracker
Where -p means recursively create directories.
After configuring tracker1 and tracker2, let's start the two trackers now. We first go to the /fastdfs/tracker/ directory we just created, and find that there is nothing in the current directory.
Before starting, we need to configure the firewall on the two devices, tracker and tracker, and add port 22122, so that other devices can access port 22122. Added content: -A INPUT -m state --state NEW -m tcp -p tcp --dport 22122 -j ACCEPT, as shown in the figure below (note that all examples are based on one example, and other operations can be followed up).
Restart the firewall after adding
service iptables restart
Below we use the command:
/etc/init.d/fdfs_trackerd start
Start it. After starting, we look at the directory and find that there are two more directories data and logs. We can use the command:
ps -ef | grep fdfs
To check whether the tracker starts normally, as shown in the figure below.
If you want to stop the tracker, use the command:
/etc/init.d/fdfs_trackerd stop
13. Configure storage. According to our plan, four virtual machines 192.168.0.252, 192.168.0.253, 192.168.0.254, and 192.168.0.249 will be used as our storage nodes. The configuration of the same group of the four nodes should be the same, and It is a group of 192.168.0.252 and 192.168.0.253 (the group name is group1), and a group of 192.168.0.254 and 192.168.0.249 (the group name is group2).
First, enter the configuration file directory /etc/fdfs of the 4 machines, then copy storage.conf.sample and rename it storage.conf, using the command:
cp storage.conf.sample storage.conf
As shown below
Next, we edit the storage.conf file. For the two devices 192.168.156.7 and 192.168.156.8 of group1, the configurations that need to be modified are as follows.
base_path=/fastdfs/storage
store_path0=/fastdfs/storage
tracker_server=192.168.0.250:22122
tracker_server=192.168.0.251:22122
After modification, as shown in the following figure:
Notice:192.168.0.252 and 192.168.0.253 belong to group1, 192.168.0.254 and 192.168.0.249 belong to group2
Since the base_path=/fastdfs/storage and store_path0=/fastdfs/storage of the four configuration files are configured as /fastdfs/storage, but the directory has not been created for these four virtual machines at present, so we need to create a directory for these four Create the directory for the virtual machine, command:
mkdir -p /fastdfs/storage
Before starting the storage, we need to configure the firewalls of the four virtual machines, add port 23000, and add the statement: -A INPUT -m state --state NEW -m tcp -p tcp --dport 23000 -j ACCEPT, As shown below.
Restart the firewall:
service iptables restart
After configuring the firewall, we go to the /fastdfs/storage directory of the four virtual machines where the storage is located, and use the command:
/etc/init.d/fdfs_storaged start
Start it, then use the command:
tailf logs/storaged.log
To view the startup information, as shown in the figure below (the operation diagram of the virtual machine 192.168.0.252). It can be seen that the device 192.168.0.252 is successfully connected to two tracker devices, and 192.168.0.251 is elected as the leader of the tracker cluster. The two virtual machines 192.168.0.252 and 192.168.0.253 belong to the same group (group1), so you can see the successful connection information with 192.168.0.253 from the information in the figure below.
There is no problem in starting 4 storage units in sequence.
14. After the cluster starts, we then test the high availability of the cluster's tracker. Now we artificially turn off the tracker of 192.168.0.251 and execute the following command:
/etc/init.d/fdfs_trackerd stop
After turning off the tracker of 192.168.0.251, let's look at the log information of the four virtual machines where the storage is located. As shown in the figure above, you can see that 192.168.0.250 has automatically switched to the leader role, and it shows that it cannot connect to 192.168.0.251 (because the tracker has been turned off, so it cannot be connected).
After we start the tracker of 192.168.0.251 again, look at the log information of the four devices, as shown in the figure below, we can see that they are all reconnected to the tracker. It's just that the leader of the tracker is still 192.168.0.250 at this time.
15. After all our tracker and storage nodes are successfully started, we can view the information of the storage cluster on any storage node, command:
/usr/bin/fdfs_monitor /etc/fdfs/storage.conf
You can see the cluster information as shown in the figure below. It can be seen that there are two tracker servers, the current service is 192.168.0.250, the number of groups is 2, the IPs of the first group are 192.168.0.252 and 192.168.0.253, and the IPs of the second group are 192.168.0.254 and 192.168. 0.249, exactly consistent with our planned cluster.
- You can use the command:
cd /usr/bin/ && ls | grep fdfs
To view all the commands of fdfs, as shown in the figure below.
17. Set FastDFS tracker and storage to start automatically
vim /etc/rc.d/rc.local
4. File upload test
1. Modify the client configuration file in the Tracker server
Execute the following command:
cp /etc/fdfs/client.conf.sample /etc/fdfs/client.conf
Modify the client configuration file:
vim /etc/fdfs/client.conf
base_path=/fastdfs/tracker
tracker_server=192.168.0.250:22122
tracker_server=192.168.0.251:22122
2. Execute the following file upload instructions:
/usr/bin/fdfs_upload_file /etc/fdfs/client.conf /usr/local/software/FastDFS_v5.08.tar.gz
Return ID number:
group1/M00/00/00/wKgBh1Xtr9-AeTfWAAVFOL7FJU4.tar.gz
group2/M00/00/00/wKgBiVXtsDmAe3kjAAVFOL7FJU4.tar.gz
(The above file ID can be returned, indicating that the file upload is successful)