Centos7rsync remote synchronization

Experimental materials:
Two Centos7s,
one serving as rsync server: 192.168.1.10 and
one serving as client: 192.168.1.110

Configure rsync source server

1. Introduction to rsync tool:
an open source fast backup tool that can mirror and synchronize the entire directory tree between different hosts, supports incremental backups, maintains links and permissions, uses optimized synchronization algorithms, performs compression before transmission, and is suitable for remote backups. Mirror server and other applications (one of the basic components installed by default in the Linux system)
2. Server identity distinction:

Initiator: The client responsible for initiating rsync synchronization operations (read permission to the document location of the synchronization source)
Synchronization source: Responsible To respond to the server (responsible for providing the original location of the document)
request from the client rsync synchronization operation : Back up the /var/www/html directory of server A to server B with the backup account ajbn to
achieve the steps:
(1), configuration synchronization Source
1. Install httpd, prepare /var/www/html directory

[root@A-Server ~]# systemctl stop firewalld       （关闭防火墙）
[root@A-Server ~]# setenforce 0			  （临时关闭selinux）
[root@A-Server ~]# mount /dev/cdrom /media/cdrom  （挂光盘）
[root@A-Server ~]# yum -y install httpd           （安装httpd服务）
[root@A-Server ~]# cd /var/www/html/		  （进入网站根目录）
[root@A-Server html]# touch aaa bbb               （创建两个文件）

2. Create the /etc/rsyncd.conf configuration file

[root@A-Server html]# vim /etc/rsyncd.conf
use chroot = yes      #禁锢在源目录
address = 192.168.1.10    #监听地址
port = 873    #监听端口
log file = /var/log/rsyncd.log   #日志文件
pid file = /var/run/rsyncd.pid   #进程文件
hosts allow = 192.168.1.0/24	 #允许访问的客户机地址

[aaa]      #共享模块名
path = /var/www/html   #需要同步的实际路径
comment = ajbn	       #描述信息，随便写
read only = yes        #只读
dont compress = *.gz *.tgz *.zip *.z *.Z *.rpm *.deb *.bz2   #同步时不在压缩的文件类型
auth users = ajbn      #授权同步的账户
secrets file = /etc/rsync_users.db   #账户所在的数据文件

3. Create a backup account data file

[root@A-Server html]# vim /etc/rsync_users.db  #创建授权账户文件
[root@A-Server html]# cat /etc/rsync_users.db
ajbn:123.com   （ajbn用户名；123.com密码）
[root@A-Server html]# chmod 600 /etc/rsync_users.db （对用户的数据文件进行权限保护）

4. Start the rsync service program

[root@A-Server html]# rsync --daemon    （启用）
[root@A-Server html]# netstat -anpt | grep 873   （查看监听状态）

Use rsync tool (for synchronous backup)

Local backup: similar to the cp command: Example: rsync /etc/fstab /opt (synchronize the local file /etc/fstab to the /opt directory)
基本语法： rsync 选项 源位置 目标位置

Options	effect
-a	Archive mode, retaining file permissions, attributes and other information is equivalent to the combined option "-rlptgoD"
-v	Show detailed synchronization process
-H	Keep hard link files
-A	Keep ACL attribute information
-with	Compress during transmission
–delete	Delete files in the target location but not in the original location
–checksum	Decide whether to skip files based on the checksum

Backup test on the client:

[root@B-Clinet ~]# systemctl stop firewalld  （关闭防火墙）
[root@B-Clinet ~]# setenforce 0    （临时关闭selinux）
[root@B-Clinet ~]# rsync -avz ajbn@192.168.1.10::aaa /root  （远程同步）
[root@B-Clinet ~]# ls   （查看）

Automatic backup combined with crond scheduled tasks

vim /etc/server.pass
123.com    （所写的就是ajbn 的密码）
[root@B-Clinet ~]# chmod 600 /etc/server.pass   （对用户的数据文件进行权限保护）
[root@B-Clinet ~]# mkdir /myweb			（创建备份目录）
[root@B-Clinet ~]# crontab -e			 (创建计划任务)
30 22 * * * /usr/bin/rsync -az --delete --password-file=/etc/server.pass ajbn@192.168.1.10::aaa /myweb
[root@B-Clinet ~]# crontab -l     （查看计划任务）
[root@B-Clinet ~]# date -s "2020-09-24 22:29"    （设置时间，方便测试）
[root@B-Clinet ~]# systemctl restart crond	 （重启服务）
[root@B-Clinet ~]# date		（查看当前时间）
[root@B-Clinet ~]# ll /myweb/	（查看验证）

Configure inotify + rsync real-time synchronization

(1) Insufficient regular backup
1. Fixed backup time, obvious delay and poor real-time performance.
2. If the backup source does not change for a long time, regular backup is a serious waste of resources.
(2) Solution: Real-time synchronization
1. Just source Once there is a change, start synchronization immediately.
2. If the source has not changed, no synchronization backup will be performed

. Inotify
inotify: Used to monitor various changes of the file system, such as file access, deletion, movement, modification, etc., which can be convenient Realize file change alarms, incremental backups, and respond to changes in directories or files in a timely manner
inotify + rsync combination: to achieve triggered backup (real-time synchronization)-as long as the original location of the document changes, immediately start incremental backup, otherwise In a silent waiting state
Case:
Server A: 192.168.1.10 Initiator, install inotify
Server B: 192.168.1.110 Synchronization source
Requirements: As soon as the data of server A changes, it will be synchronized to the
inotify source package on server B :
https:// pan.baidu.com/s/150e64cB7dBIi0EaZqvqtFA
Extraction code: j4jf
Steps:
(1) Adjust the inotify kernel parameters (monitoring event queue); maximum number of monitoring instances; maximum number of monitoring files per instance
192.168.1.10：
Increase the values ​​of three parameters: Monitoring The number of directories and files is large or changes frequently

（查询默认的监控事件队列）
[root@A-Server ~]# cat /proc/sys/fs/inotify/max_queued_events
（查询默认的最多实例数）
[root@A-Server ~]# cat /proc/sys/fs/inotify/max_user_instances 
（查询每个实例最多监控的文件数）
[root@A-Server ~]# cat /proc/sys/fs/inotify/max_user_watches 

Tuning:

[root@A-Server ~]# vim /etc/sysctl.conf
添加：
fs.inotify.max_queued_events = 16384
fs.inotify.max_user_instances = 1024
fs.inotify.max_user_watches = 1048576
[root@A-Server ~]# sysctl -p  （让其立即生效）

(2) Install inotify-tools software; in order to provide inotifywait and inotifywatch auxiliary tool programs
192.168.1.10：

[root@A-Server ~]# tar zxf inotify-tools-3.14.tar.gz 
[root@A-Server ~]# cd inotify-tools-3.14/
[root@A-Server inotify-tools-3.14]# ./configure && make && make install

Test: monitor changes in the /mnt directory

[root@A-Server ~]# inotifywait -mrq -e modify,create,move,delete  /mnt

Monitoring options	effect
-m	Indicates continuous monitoring
-r	Means recurse the entire directory
-q	Simplify output messages
-e	Specify which write events to monitor
modify
create	create
move	mobile
delete	delete
attrib	Attribute changes

在打开一个192.168.1.10的终端
[root@A-Server ~]# cd /mnt/  （进入/mnt目录）
[root@A-Server mnt]# ls      （查看是否有数据）
[root@A-Server mnt]# touch a.txt b.txt   （创建两个文件）

回来原来的监控终端进行查看

(Three), write trigger synchronization script
192.168.1.10：

[root@A-Server ~]# vim /opt/tb.sh
#!/bin/bash
AAA="inotifywait -mrq -e modify,create,attrib,move,delete /mnt"
BBB="rsync -azH --delete --password-file=/etc/server.pass /mnt [email protected]::aaa"
$AAA | while read DIRECTORY EVERT FILE 
do
$BBB
done

Create a password file:

[root@A-Server ~]# vim /etc/server.pass
123.com   （同步源的密码）
[root@A-Server ~]# chmod 600 /etc/server.pass  （对用户的数据文件进行权限保护）

(4) Configure the synchronization source
192.168.1.110：
1. Establish the /etc/rsyncd.conf configuration file

[root@B-Client ~]# vim /etc/rsyncd.conf
use chroot = yes
address = 192.168.1.110
port = 873
log file = /var/log/rsyncd.log
pid file = /var/run/rsyncd.pid
hosts allow = 192.168.1.0/24
fake super = yes

[aaa]
path = /mnt
comment = ajbn
read only = no
dont comperss = *.gz *.tgz *.zip *.z *.Z *.rpm *.deb *.bz2
auth users = ajbn
secrets file = /etc/rsync_users.db
[root@B-Client ~]# chmod 777 /mnt/

2. Create a backup account data file

[root@B-Client ~]# vim /etc/rsync_users.db
[root@B-Client ~]# cat /etc/rsync_users.db
ajbn:123.com     （ajbn 用户名； 123.com 密码）
[root@B-Client ~]# chmod 600 /etc/rsync_users.db  （对用户的数据文件进行权限保护）
[root@B-Client ~]# rsync --daemon   （启用）
[root@B-Client ~]# netstat -anpt | grep 873  （查看rsync服务状态）

3. Start the rsync service program

[root@B-Client ~]# rsync --daemon     （启用）
[root@B-Client ~]# netstat -anpt | grep 873  （查看端口号）

(5) Trigger the script on the initiator
192.168.1.10：

把脚本添加到自运行中执行脚本：
[root@A-Server ~]# echo '/opt/tb.sh' >> /etc/rc.local  
后台运行该脚本：
[root@A-Server ~]# .  /opt/tb.sh &

(6), verify
cd /mnt (create file) on the 192.168.1.10 machine

[root@A-Server ~]# cd /mnt/  （进入/mnt目录）
[root@A-Server mnt]# ls	     （查看）
a.txt  b.txt                 （这是原有文件）
[root@A-Server mnt]# touch ajbn    （创建一个文件）
[root@A-Server mnt]# touch ajbm    （再创建一个文件）
[root@A-Server mnt]# ls	           （进行查看）
ajbm  ajbn  a.txt  b.txt

Cd /mnt on the 192.168.1.110 machine (found that the data inside is synchronized with 192.168.1.10)

[root@B-Client ~]# cd /mnt/  （进入/mnt目录）
[root@B-Client mnt]# ls	     （查看）
mnt			     （发现有个子目录）
[root@B-Client mnt]# cd mnt/ （进入子/mnt目录）
[root@B-Client mnt]# ls	     （再次进行查看）
ajbm  ajbn  a.txt  b.txt     （数据已经同步）