rsync remote synchronization + real-time synchronization

rsync overview

rsync, the full name: Remote Sync (remote synchronization), is an open source fast incremental backup tool that can mirror and synchronize the entire directory tree between different hosts. It also supports local replication, incremental backup, connection and permissions, or Synchronize with other SSH and rsync hosts, adopt optimized synchronization algorithm, and perform compression before transmission, so it is very suitable for remote backup, mirror server and other applications

rsync command

rsync 选项 源位置 目标位置

常见选项
-a 归档模式，递归保留对象属性
-v 显示同步过程
-z 传输时进行压缩
-H 保留硬连接文件
-A 保留ACL访问控制列表信息
–delete 删除目标位置有的，源没有的文件，可以使文件同步
–checksum 根据对象的校验来决定是否跳过文件

Two representation methods of configuration source

格式1:用户名@主机地址::共享模块名
格式2:rsync:/l用户名@主机地址/共享模块名

rsync synchronization

[root@localhost ~]# rsync --version		#查看rsync版本，系统自带，不用安装
rsync  version 3.1.2  protocol version 31
[root@localhost ~]# vim /etc/rsyncd.conf  #编辑主配置问啊进
uid = nobody
gid = nobody
use chroot = yes	#禁止目录
# max connections = 4
port 873
log file = /var/log/rsync.log
host allow = 20.0.0.0/32
pid file = /var/run/rsyncd.pid

[CSDN]
path = /var/www/html
comment = www.pan.com
read only = yes
dont compress = *.rar *.gz *.zip *.tgz *.z *.Z *.rpm *.deb *.bz2	#不解压的文件格式
auth users = juejue
secrets file = /etc/rsyncd_users.db

[root@localhost ~]# vim /etc/rsyncd_users.db	#文件名、位置和上面配置文件的一致
pan:123123
[root@localhost ~]# chmod 600 /etc/rsyncd_users.db
[root@localhost ~]# rsync --daemon
[root@localhost ~]# netstat -ntapu |grep 873
tcp        0      0 0.0.0.0:873             0.0.0.0:*               LISTEN      14078/rsync         
tcp6       0      0 :::873                  :::*                    LISTEN      14078/rsync
[root@localhost html]# cd /var/www/html/
[root@localhost html]# echo "This is Hello" > index.html

Synchronized client

[root@localhost /]# cd /var/www/html/
[root@localhost html]# ls
[root@localhost html]# 
[root@localhost html]# rsync -avz [email protected]::CSDN /var/www/html/
Password: 	#输入刚才创建的密码
receiving incremental file list
./
index.html

sent 46 bytes  received 125 bytes  11.79 bytes/sec
total size is 14  speedup is 0.08
[root@localhost html]# ls
index.html		##同步完成

制作免密同步
[root@localhost html]# mkdir /mima
[root@localhost html]# vim /mima/pwd.pass
123123
[root@localhost html]# chmod 600 /mianmi/pwd.pass	#给600权限是为了防止除管理员用户外的用户篡改文件

verification

[root@localhost html]# echo "11" > ABC.txt	#在 源 创建一个文件


[root@localhost html]# rsync -avz --delete --password-file=/mianmi/pwd.pass [email protected]::CSDN /var/www/html/	#客户机输入，可以看到确实不用输入密码
receiving incremental file list
./
ABC.txt

sent 46 bytes  received 142 bytes  376.00 bytes/sec
total size is 20  speedup is 0.11
[root@localhost html]# ls	##同步成功
ABC.txt  index.html
[root@localhost html]# cat ABC.txt 
11

rsync real-time synchronization

• The lack of regular synchronization
  • The time to perform the backup is fixed, with obvious delay and poor real-time performance
  • When the synchronization source does not change for a long time, intensive periodic tasks are unnecessary
• Advantages of real-time synchronization
  • -Once the synchronization source changes, immediately start the backup
  • As long as there is no change in the synchronization source, no backup is performed

Inotify is a Linux kernel mechanism that can monitor file system operations, such as reading, writing, and creating.
Inotify is responsive, very simple to use, and more efficient than busy polling of cron tasks.
Available from version 2.6.13; it
can monitor changes in the file system and respond to notifications;
auxiliary software: inotify-tools

inotifywait：用于持续监控，实时输出结果
inotifywatch：用于短期监控，任务完成后再出结果

Source end arrangement

[root@localhost ~]# vim /etc/rsyncd.conf
[CSDN]
path = /var/www/html
comment = www.pan.com
read only = no		#这行改成no
[root@localhost www]# chmod 777 /var/www/html/
重启服务

Client configuration

[root@localhost ~]# cat /proc/sys/fs/inotify/max_queued_events
16384	#监控队列大小
[root@localhost ~]# cat /proc/sys/fs/inotify/max_user_instances
128		#最多监控实例数
[root@localhost ~]# cat /proc/sys/fs/inotify/max_user_watches
8192	#每个实例最多监控文件数
[root@localhost ~]# vim /etc/sysctl.conf 
fs.inotify.max_queued_events = 16348
fs.inotify.max_user_instances = 1024
fs.inotify.max_user_watches = 1048576
[root@localhost ~]# sysctl -p
fs.inotify.max_queued_events = 16348
fs.inotify.max_user_instances = 1024
fs.inotify.max_user_watches = 1048576

安装软件 inotify
[root@localhost ~]# cd /opt
[root@localhost opt]# tar zxvf inotify-tools-3.14.tar.gz
[root@localhost opt]# cd inotify-tools-3.14/
[root@localhost inotify-tools-3.14]# yum -y install gcc gcc-c++		##安装环境依赖      
[root@localhost inotify-tools-3.14]# ls
aclocal.m4    config.h.in   COPYING     libinotifytools  man      src
AUTHORS       config.sub    depcomp     ltmain.sh        missing
ChangeLog     configure     INSTALL     Makefile.am      NEWS
config.guess  configure.ac  install-sh  Makefile.in      README
[root@localhost inotify-tools-3.14]# ./configure
[root@localhost inotify-tools-3.14]# make && make install

Open another client terminal interface

[root@localhost opt]# vim inotify.sh##客户机编辑脚本监控共享目录变化并启动,方便管理
#!/bin/bash
INOTIFY_CMD="inotifywait -mrq -e create,delete,move,modify,attrib,move,delete /var/www/html/"RSYNC_CMD="rsync -azH --delete --password-file=/mianmi/pwd.pass /var/www/html/ [email protected]::CSDN /var/www/html"

$INOTIFY_CMD | while read DIRECTORY EVENT FILE
do
    if [ $(pgrep rsync | wc -l) -le 0 ] ; then
        $RSYNC_CMD
    fi
done

[root@localhost opt]# chmod 755 inotify.sh 
[root@localhost opt]# chmod 777 /var/www/html/
[root@localhost opt]# chmod +x inotify.sh 
[root@localhost opt]# ./inotify.sh

verification

第三个客户端
[root@localhost ~]# inotifywait -mrq -e modify,create,move,delete  /var/www/html
# modify 修改，create 创建，move 移动， delete 删除
/var/www/html/ DELETE DEMAXIYA.txt
/var/www/html/ CREATE DEMAXIYA.txt
/var/www/html/ MODIFY DEMAXIYA.txt

[root@localhost html]# rm -rf *
[root@localhost html]# echo "123456" >DEMAXIYA.txt

[root@localhost html]# ls
DEMAXIYA.txt