Manual operation of rsync synchronization recognition and configuration, secret-free synchronization, automatic operation, automatic synchronization with inotify+rsync
-
- One, the understanding of rsync synchronization
- Second, the configuration of rsync synchronization
- Three, faults and solutions
One, the understanding of rsync synchronization
[1] Application scenario
Rsync is a tool that can implement incremental backups. With task planning, rsync can achieve timing or interval synchronization, and with inotify or sersync, it can achieve triggered real-time synchronization.
[2] Three working modes
Synchronize on the local file system:
rsync -auv /opt/rsync/ /opt/local/
The local host uses the remote shell to communicate with the remote host (scp mode)
rsync -auv /opt/rsync/ [email protected]:/opt/test
The local host connects to the rsync daemon on the remote host through a network socket
rsync -auv /opt/rsync [email protected]::test
Description:
- 1. The previous command is the source directory, and the latter is the destination directory. Indicates that the source directory file is synchronized to the destination directory based on the source directory.
- 2. If there are more than 2 paths, then the last path is the target path, and the front is the source directory, which means that all files in the source directory are synchronized to the target path
- 3. If the source directory does not have the suffix "/", it means that the directory is created in the target directory and the files in the source directory are synchronized together, and the "/" means that all files in the source directory are synchronized.
- 4. Regarding the third mode, the server can be started as a single process or by xinetd, both of which listen on port 873. The third mode can specify the directory to be opened to the outside world and upload the directory/ Download permission control restricts the client through ip, and can set passwords and so on. Compared with xinetd, the independent process mode has greater ability to handle requests, while xinetd is lighter.
[3] Commonly used options
-n Only test transmission, not actual transmission. Often used with "-vvvv" to see how rsync works.
-auv (usually these 3 parameters are used for synchronization)
-v: Display detailed information during rsync. You can use "-vvvv" to get more detailed information
-a --archive: archive mode, which means recursive transmission and keep file attributes. Equivalent to "-rtopgDl".
-u --update: Only copy when the source mtime is newer than the mtime of the target existing file. Note that this option is judged by the receiving end and will not affect the deletion behavior
–exclude Exclude a directory/file rsync -r -v --exclude=“anaconda/*.log” /var/log/anaconda /var/log/ audit /tmp
-delete After using the "-delete" option, rsync on the receiving end will first delete files that already exist in the target directory but the source directory does not exist.
That is, "the more you delete, and the less you make up."
Note: If you use the "–delete" option and the "–exclude" option together, the excluded files will not be deleted
Second, the configuration of rsync synchronization
【1】Main service configuration of manual synchronization
yum install -y httpd
yum install -y rsync
(1) Modify configuration file information
vim /etc/rsyncd.conf
uid = nobody
gid = nobody
use chroot = yes
port 873
max connections = 4
log file = /var/log/rsyncd.log
pid file = /var/run/rsyncd.pid
hosts allow = 192.168.60.0/24
[wwwroot]
path = /var/www/html
comment = www.test.com
read only = yes
dont compress = *.gz *.tgz *.zip *.z *.Z *.rpm *.deb *.bz2
auth users = backuper
secrets file = /etc/rsyncd_users.db
(2) Write a password file
vim /etc/rsyncd_users.db
backuper:123123
chmod 600 /etc/rsyncd_users.db
(3) Start rsync
rsync --daemon //启动rsync同步服务
netstat -natp | grep rsync //查看端口状态
pkill rsync //结束进程
(4) Write web site files
cd /var/www/html
vim index.html
this is shoudong web
[2] Synchronous configuration from the server for manual synchronization
//Manual synchronization method one
rsync -avz [email protected]::wwwroot /var/www/html
//Manual synchronization method two
rsync -avz rsync://[email protected]/wwwroot /var/www/html
//Password-free login manual synchronization method three
vim /opt/server.pass
chmod 600 /opt/server.pass
rsync -zva --delete --password-file=/opt/server.pass [email protected]::wwwroot /var/www/html
[3] Automatic synchronization of inotify+rsync is configured on the initiator
(1) Set up monitoring events [the script to pave the way]
vim /etc/sysctl.conf
fs.inotify.max_queued_events = 16384 //Monitor event queue size
fs.inotify.max_user_instances = 1024 //Monitor the maximum number of instances
fs.inotify.max_user_watches = 1048576 //The maximum number of monitored files per instance , The number of monitoring should be greater than the total number of files monitored
sysctl -p
(2) Install inotify-tools auxiliary tools
yum install -y gcc gcc-c++
tar zxvf inotify-tools-3.14.tar.gz
cd /opt/inotify-tools-3.14
./configure
make && make install
(3) Test whether the inotify auxiliary tool is normal
inotifywait is used for continuous monitoring, and real-time output results
inotifywatch are used for short-term monitoring. After the task is completed, the output results are
continuously monitored
-r recursively monitor all sub-objects
-q simplified output information
-e specify which event types to monitor
inotifywait -mrq -e modify,create,move,delete /var/www/html
Also when opening a new terminal
(4) rsync+inotify real-time synchronization
//Set on the server
vim /etc/rsyncd.conf
read only = no
chmod 777 /var/www/html
//Set on the initiator
chmod 777 /var/www/html
vim /opt/inotify_rsync.sh
#!/bin/bash
INOTIFY_CMD="inotifywait -mrq -e modify,create,move,delete /var/www/html"
RSYNC_CMD="rsync -avz --delete --password-file=/opt/server.pass [email protected]::wwwroot /var/www/html"
$INOTIFY_CMD | while read DIRECTORY EVENT FILE
do
if [ $(pgrep rsync |wc -l) -le 0 ];then
$RSYNC_CMD
fi
done
sh inotify.sh
(5) Verification
Create and write content on the initiator
[root@promote html]# echo "this is abc web" >> abc.html to
see if the monitoring script is normal [the error below does not matter]
Check whether it is synchronized on the server
Three, faults and solutions
[Question 1] A compilation error occurred when compiling and installing the inotify-tools tool
Analysis: configure: error: no acceptable C compiler found in $PATH error is mainly because there is no C compiler.
[Solution] Install the C compiler:
[root@localhost ~]# yum -y install gcc gcc-c++
[Question two] No effect after executing the script
Analysis: This result indicates that the script information was not executed, and the problem occurred in the content of the script. After analysis, the execution of the script lies in the if conditional statement, and the most important thing is pgrep rsync | wc -l.
Execute pgrep rsync | wc -l to check that if it is greater than 0, all rsync processes exist.
[Solution] Shut down the rsync process on the initiator
[root@localhost opt]# pkill rsync
[root@localhost opt]# netstat -natp | grep rsync