Incremental file synchronization between linux and windows servers: Rsync usage tutorial
For operation and maintenance personnel, data backup is an important and necessary daily work. Choosing a good backup software will greatly improve the efficiency. rsync is a powerful, efficient, secure, and fast incremental file transfer tool.
1. What is rync?
- rsync ---- remote synchronize, is a software to realize remote synchronization function;
- rsync uses the "Rsync algorithm" to synchronize files, which only transfers the different parts of the two files, so it is quite fast;
- While synchronizing files, the permissions, time and directory structure of the original files can be maintained;
- For multiple files, the internal pipeline reduces the delay of file waiting;
- rsync listens on TCP port 873 by default, and copies files through remote shells such as rsh and ssh. It is also required that sync must be installed on both the remote and local systems.
Two, preparation
Source file address: 192.168.0.1 centos7
target file address: 192.168.0.2 windows 10
Please turn off the firewall: systemctl stop firewalld.service
Check the firewall status: firewall-cmd –state
Please set the value of SELINUX in the /etc/sysconfig/selinux file to disable ,Restart linux after the modification to make the modification take effect, otherwise it will affect the file synchronization.
The effect achieved by the following steps:
the content under the folder to be synchronized in the client (application server, file source server) 192.168.0.1 (/usr/lmr/backup/ ) to the server (backup server, target server) 192.168.0.2 (E:\desktop\new folder).
3. Install and configure the Rsync server (linux)
1. Check if rsync is installed
installed
not installed
2. Install rsync
At present, rsync has been installed in all major distribution systems of Llinux. You can view the current version through rysnc --version. If you need to install the latest version, you can go to the official website of rysnc: http://rsync.samba.org/ to download the latest version , compile and install.
#yum install -y rsync ####yum 安装
Compile and install:
#wget https://download.samba.org/pub/rsync/src/rsync-3.1.3.tar.gz #tar -zxvf rsync-3.1.3.tar.gz #cd rysnc-3.1.3 #./configure --prefix=/usr/local/rsync #make && make install
The rsync installation is complete, and it is under /usr/local/rsync/bin by default. After configuration, it can back up or transfer remote server data.
Note: The gcc compilation tool must be installed before compiling and installing.
3. Configure rsync
- The configuration file
rsync mainly has the following three configuration files:
rsyncd.conf ---- main configuration file, need to manually generate
rsyncd.pass ---- password file
rsyncd.motd ---- rysnc server information
The address where I save the rsync configuration file is: /usr/lmr/soft/rsync/
First enter the folder where the configuration file is to be stored
ln -s /etc/rsyncd.conf #关联rsync的默认配置文件，使我们的配置文件生效，会创建rsyncd.conf文件
vi rsyncd.conf
rsyncd.conf (the main configuration file of rsync service, which controls various attributes of rsync service, an example of rsyncd.conf file is given below
#进行同步或者备份的用户，nobody 为任何用户 uid = root #进行备份的组，nobody为任意组 gid = root #端口 port = 873 #如果"use chroot"指定为true，那么rsync在传输文件以前首先chroot到path参数所指定的目录下。这样做的原因是实现额外的安全防护，但是缺点是需要以root权限，并且不能备份指向外部的符号连接所指向的目录文件。默认情况下chroot值为true.但是一般不需要，选择no或false use chroot = yes read only = on #不允许列清单 list = no #最大连接数 max connections = 4 #pid文件的存放位置 #pidfile = /var/run/rsyncd.pid #锁文件的存放位置 lock file=/var/run/rsyncd.lock #日志文件的存放位置 log file = /var/log/rsyncd.log motd file = /etc/rsyncd/rsyncd.motd # //此文件定义完成后系统会自动创建 exclude = lost+found/ transfer logging = yes #覆盖客户指定的IP超时时间，也就是说rsync服务器不会永远等待一个崩溃的客户端。 timeout = 900 ignore nonreadable = yes # //同步时跳过没有权限的目录 dont compress = *.gz *.tgz *.zip *.z *.Z *.rpm *.deb *.bz2 # //传输时不压缩的文件 #哪些电脑可以访问rsync服务，这里可以指定单个IP，也可以指定整个网段，能提高安全性。格式是ip 与ip 之间、ip和网段之间、网段和网段之间要用空格隔开 #hosts allow = 172.25.0.110 #哪些电脑不可以访问rsync服务 #hosts deny = 172.25.0.0/24 #这里是认证模块名，即跟samba语法一样，是对外公布的名字 [backup] comment = this is module for backup #这里是参与同步的目录 path = /usr/lmr/backups/ #可以忽略一些无关的IO错误 ignore errors #允许可读可写 read only = no #认证的用户名 auth users = root #密码文件存放地址 secrets file = /etc/rsyncd.pass
Note: auth users = user name for redhat authentication. This name is a real user on the server side. Don't follow the steps directly but ignore this point. If this is missing on the server side, I guess your data synchronization will not be possible, everyone should keep in mind
Write a user password file:
echo "root:123" > /etc/rsyncd.pass
root: the login user name set in the previous step; 123: the password of the file, which can be set at will; and the rsyncd.pass file is created in this step to
modify the password file permission
chmod 600 /etc/rsyncd.pass
Define rsyncd.motd file
rsyncd.motd mainly defines the welcome information for users to log in to the rsync service. You can define it according to your needs. It is the same as the FTP login interface:
# vi /etc/rsyncd.motd
*****************************************************
Welcome to use the zxl's rsync services!
*****************************************************
Start the rsync service
#/usr/bin/rsync --daemon
#netstat -lntup |grep rsync
#ps -ef |grep rsync |grep -v grep
Port 873 appears to indicate successful startup
Set up autostart
#echo "/usr/bin/rsync --deamon">>/etc/rc.local #cat /etc/rc.local
Note: –daemon is to make rsync run in server mode
Note: The inability to access may be a firewall problem, you can open port 873, please Baidu for specific methods
4. Install the rsync client (windows)
1. Download and install
Client download [cwRsync_4.1.10_client_windows.zip]
https://www.jb51.net/softs/55934.html#downintro2
After installation, as shown in the figure below, the red one is a new password.txt
The content of password.txt is as follows
123
That is the same as the password of rsync.passwd on the corresponding centos
2. Execute the command
Note: Here you need to enter the bin directory of the rsync installation directory to execute
./rsync.exe -avzP --port=873 --password-file=/cygdrive/e/软件/cwRsync/bin/password.txt [email protected]::backup /cygdrive/e/桌面/新建文件夹
You can also write directly like this:
"E:\软件\cwRsync\bin\rsync.exe" -avzP --port=873 --password-file=/cygdrive/e/软件/cwRsync/bin/password.txt [email protected]::backup /cygdrive/e/桌面/新建文件夹
The meaning of this command is simply stated
– port=873 #Port
root #User who performs data synchronization
192.168.0.1 #Server address
backup #Module name, which is [backup]
/cygdrive/e/desktop/ in the rsync server configuration file New folder #Indicates the local synchronization folder, /cygdrive/e/ is equivalent to e:/
if you specify
–delete #Delete the Rsync server /usr/lmr/ from the cwRsync client directory e:/desktop/new folder Different data in the backup directory, that is, to ensure that the data on both sides are completely consistent
/cygdrive/e/software/cwRsync/bin/password.txt #Password file
When running, the effect of synchronization is as follows
5. Timing backup
1. Linux scheduled backup
Combined with contab, scheduled backup
Create a scheduled backup task, and synchronize the /home/data file to be backed up on host A to the /backup directory under host B (172.25.0.150).
Execute the backup task every night at 23:30
vim AtoBbacku.sh #!/bin/bash rsync -avzP --delete --password-file=rsyncd.secrets /home/data [email protected]::/backup
chmod 755 AtoBbacku.sh
crontab -e
30 23 * * * sh -x /AtoBbacku.sh
2. windows backup
1> Create a new rysnc.txt file and fill in the following content
@echo off
"E:\软件\cwRsync\bin\rsync.exe" -avzP --port=873 --password-file=/cygdrive/e/软件/cwRsync/bin/password.txt [email protected]::backup /cygdrive/e/桌面/新建文件夹
Save this file as rysnc.bat file, encoded as ANSI
Note: If the encoding is utf8, the Chinese in the text will appear garbled and cause execution errors.
2> Turn on the scheduled task
After the batch processing is completed, how to run the script periodically? Windows comes with a very powerful scheduled task function. Enter computer management (right click on this computer and "manage"), you can see all the scheduled tasks of your computer in the system tool -> task scheduler -> task scheduler library, right click to create a basic task, enter the name and description as shown in the figure
, Click Next to set the trigger (task trigger time).
Click Next to set the task execution operation. Here, select the startup program.
Select the bat script you just made, and click Next.
Click Finish to create the task.
Select the task you just created, and right-click Properties to open the Properties dialog box. , click the trigger and double-click the set trigger to set the task to execute repeatedly
Click OK to complete the scheduled task settings, right-click the task to start the task.
Six, rsync six different working modes:
1. Copy the local files, copy the files in the /home/coremail directory to the /cmbak directory
$ rsync -avSH /home/coremail/ /cmbak/
2. Copy the content of the local machine to the remote machine
$ rsync -of /home/coremail/ 192.168.11.12:/home/coremail/
3. Copy the contents of the remote machine to the local machine
$ rsync -of 192.168.11.11:/home/coremail/ /home/coremail/
4. Copy the files of the remote rsync server (running rsync in daemon form) to the local machine
$ rsync -by [email protected] ::www /databack
5. Copy the local machine files to the remote rsync server (running rsync in daemon form). This mode is enabled when the DST path information contains the "::" separator
$ rsync -off /databack [email protected] ::www
6. Display the file list of the remote machine. This is similar to rsync transmission, but just omit the local machine information in the command
$ rsync -v rsync://192.168.11.11/data
link: http://luomuren.top/articles/2021/04/06/1617641017252.html