FastDFS high availability cluster deployment and installation

1. Environmental information:

server	deployment service
16.32.15.200	Tracker (scheduling work), Storage (storage), Nginx, Keepalived
16.32.15.201	Tracker (scheduling work), Storage (storage), Nginx, Keepalived
16.32.15.202	The VIP address of the above two

2. Deploy FastDFS

Normal deployment FastDFSis omitted here, refer to: FastDFS single-node deployment and installation

3. Tracker node configuration

Mainly check the following configuration items, the configuration of the two servers is consistent

cat /etc/fdfs/tracker.conf

# 是否启用配置文件,false表示生效
disabled=false     
# 提供服务端口号
port=22122   
# 存储数据和日志目录
base_path=/home/fastdfs/tracker
# 0:轮循 1:指定组 2:负载平衡
store_lookup=2 

Start the tracker node, and the two servers operate synchronously

fdfs_trackerd /etc/fdfs/tracker.conf restart
netstat -anpt |grep 22122

4. Storage node configuration

Mainly check the following configuration items, the configuration of the two servers is consistent

cat /etc/fdfs/storage.conf

# 是否启用配置文件,false表示生效
disabled=false						
group_name=group1    				        
port=23000						
base_path=/home/fastdfs/storage	                      
store_path0=/home/fastdfs/storage

# 存储路径个数,需要和store_path个数匹配
store_path_count=1
# tracker节点IP+端口
tracker_server=16.32.15.200:22122
tracker_server=16.32.15.201:22122
# http端口号
http.server_port=8888				

Start the storage node, and the two servers operate synchronously

fdfs_storaged /etc/fdfs/storage.conf restart
netstat -anpt |grep 23000

5. View cluster information

fdfs_monitor /etc/fdfs/storage.conf

[2023-02-27 19:50:16] DEBUG - base_path=/home/fastdfs/storage, connect_timeout=30, network_timeout=60, tracker_server_count=2, anti_steal_token=0, anti_steal_secret_key length=0, use_connection_pool=0, g_connection_pool_max_idle_time=3600s, use_storage_id=0, storage server id count: 0

server_count=2, server_index=0

tracker server is 16.32.15.200:22122

group count: 2

Group 1:
group name = group1
disk total space = 27626 MB
disk free space = 14349 MB
trunk free space = 0 MB
storage server count = 2
active server count = 2
storage server port = 23000
storage HTTP port = 8888
store path count = 1
subdir count per path = 256
current write server index = 0
current trunk file id = 0

        Storage 1:
                id = 16.32.15.200
                ip_addr = 16.32.15.200 (fdfs.huayunworld.com)  ACTIVE
                http domain = 
                version = 5.05
                join time = 2023-02-27 17:55:49
                up time = 2023-02-27 19:49:53
                total storage = 27626 MB
                free storage = 14349 MB
                upload priority = 10
                store_path_count = 1
                subdir_count_per_path = 256
                storage_port = 23000
                storage_http_port = 8888
                current_write_path = 0
                source storage id = 16.32.15.201
                if_trunk_server = 0
                connection.alloc_count = 256
                connection.current_count = 0
                connection.max_count = 0
                total_upload_count = 2
                success_upload_count = 2
                total_append_count = 0
                success_append_count = 0
                total_modify_count = 0
                success_modify_count = 0
                total_truncate_count = 0
                success_truncate_count = 0
                total_set_meta_count = 0
                success_set_meta_count = 0
                total_delete_count = 0
                success_delete_count = 0
                total_download_count = 0
                success_download_count = 0
                total_get_meta_count = 0
                success_get_meta_count = 0
                total_create_link_count = 0
                success_create_link_count = 0
                total_delete_link_count = 0
                success_delete_link_count = 0
                total_upload_bytes = 233900
                success_upload_bytes = 233900
                total_append_bytes = 0
                success_append_bytes = 0
                total_modify_bytes = 0
                success_modify_bytes = 0
                stotal_download_bytes = 0
                success_download_bytes = 0
                total_sync_in_bytes = 9
                success_sync_in_bytes = 9
                total_sync_out_bytes = 0
                success_sync_out_bytes = 0
                total_file_open_count = 3
                success_file_open_count = 3
                total_file_read_count = 0
                success_file_read_count = 0
                total_file_write_count = 3
                success_file_write_count = 3
                last_heart_beat_time = 2023-02-27 19:49:53
                last_source_update = 2023-02-27 18:33:28
                last_sync_update = 2023-02-27 18:29:25
                last_synced_timestamp = 2023-02-27 18:29:17 (-1s delay)
        Storage 2:
                id = 16.32.15.201
                ip_addr = 16.32.15.201  ACTIVE
                http domain = 
                version = 5.05
                join time = 2023-02-27 17:55:48
                up time = 2023-02-27 19:49:49
                total storage = 27626 MB
                free storage = 22996 MB
                upload priority = 10
                store_path_count = 1
                subdir_count_per_path = 256
                storage_port = 23000
                storage_http_port = 8888
                current_write_path = 0
                source storage id = 
                if_trunk_server = 0
                connection.alloc_count = 256
                connection.current_count = 0
                connection.max_count = 0
                total_upload_count = 1
                success_upload_count = 1
                total_append_count = 0
                success_append_count = 0
                total_modify_count = 0
                success_modify_count = 0
                total_truncate_count = 0
                success_truncate_count = 0
                total_set_meta_count = 0
                success_set_meta_count = 0
                total_delete_count = 0
                success_delete_count = 0
                total_download_count = 0
                success_download_count = 0
                total_get_meta_count = 0
                success_get_meta_count = 0
                total_create_link_count = 0
                success_create_link_count = 0
                total_delete_link_count = 0
                success_delete_link_count = 0
                total_upload_bytes = 9
                success_upload_bytes = 9
                total_append_bytes = 0
                success_append_bytes = 0
                total_modify_bytes = 0
                success_modify_bytes = 0
                stotal_download_bytes = 0
                success_download_bytes = 0
                total_sync_in_bytes = 233900
                success_sync_in_bytes = 233900
                total_sync_out_bytes = 0
                success_sync_out_bytes = 0
                total_file_open_count = 3
                success_file_open_count = 3
                total_file_read_count = 0
                success_file_read_count = 0
                total_file_write_count = 3
                success_file_write_count = 3
                last_heart_beat_time = 2023-02-27 19:49:52
                last_source_update = 2023-02-27 18:29:16
                last_sync_update = 2023-02-27 18:33:33
                last_synced_timestamp = 2023-02-27 18:33:29 (-1s delay)

Group 2:
... .... ...

6. nginx module configuration, the configuration of the two servers is consistent

cat /etc/fdfs/mod_fastdfs.conf 

base_path=/home/fastdfs/storage
store_path0=/home/fastdfs/storage
tracker_server=16.32.15.200:22122
tracker_server=16.32.15.201:22122
storage_server_port=23000

7. Upload files to check whether the two nodes are synchronized

cat /etc/fdfs/client.conf

base_path=/home/fastdfs/storage
tracker_server=16.32.15.200:22122
tracker_server=16.32.15.201:22122

fdfs_upload_file /etc/fdfs/mod_fastdfs.conf [上传文件名]

9. Two servers install nginx service and configure keepalived virtual address

Omit... ... The final implementation is as follows:

• 16.32.15.200 Nginx master
• 16.32.15.201 Nginx backup
• 16.32.15.202 keepalived virtual address

10. FastDFS high availability verification

upload image file

fdfs_upload_file /etc/fdfs/mod_fastdfs.conf /root/jg.png

Browser access picture: http://IP:Port/group1/M00/00/00/ECAPyGP8n2aAfcY6AAONqU5ReXo078.png

At this moment, perform active-standby switchover, switch the virtual IP address to the standby machine, and then access