Rsync timing synchronization + remote backup

Introduction : Rsync is a data backup tool under unix system. It can realize the copying of local files and the synchronous transmission of remote files.

Features:

1. Rsync is easy to install and easy to use.

2. Compress and decompress in the transmission process to reduce bandwidth occupation.

3. Can update the whole file number.

4. Safe, can use rsh, ssh or direct port as transmission port, or socket connection.

5. The highlight is free.

Simply put, the purpose of Rcync is to achieve file synchronization on two hosts. (Including two ways of pushing remote files locally and pulling local files remotely)

Uh huh, the file synchronization of the two hosts involves the source server and the target server.

Based on the file you want to synchronize, the file to be synchronized is on the A server, and the A server is used as the source server. The B server is the target server.

Understanding the primary and secondary, the demand comes: we need to synchronize a certain file on the A server to a certain folder on the B server, how to achieve it?

Take Centos as an example:

1. Enter rsync to check whether the tool is installed in the system.

2. Install it manually if it is not installed.

yum install rsync -y

expand

rsync has a local file copy function.

Local file synchronization is similar to the copy command cp, which synchronizes folder a to folder b

rsync -源文件夹 源路径  目标路径
rsync -a /tmp/a/  /tmp/b

Parameters: -a archive -av archive and deduplication

1. Realize file synchronization through ssh ip direct connection.

A password is required to connect via ssh:

rsync -av -e "ssh -p 端口号" 源文件夹  目标用户@目标ip:目标文件夹（将a文件夹的文件同步到c文件夹下）
rsync -av -e "ssh -p 22" /tmp/a/  [email protected]:/tmp/c

2. Realize file synchronization in the form of module components.

Rsync can be used as a server to connect remotely with other Rsync components

After installing Rsync on the source server, start

yum -y install rsync xinetd
两种启动方式：
(1)独立启动 /usr/bin/rsync –-daemon
(2)用xinetd超级进程启动/etc/rc.d/init.d/xinetd reload

扩展：
rsync默认端口：837 （注意防火墙是否拦截）
/usr/bin/rsync --daemon  --config=/etc/rsyncd/rsyncd.conf 　#--config用于指定rsyncd.conf的位置,如果在/etc下可以不写

1. Not password protected

1) Write configuration file:

#创建 rsyncd.conf，这是 rsync 服务器的配置文件
vi /etc/rsyncd.conf 

#模块名
[mag_sync]
#源文件路径
path = /usr/local/mongodb/data/auditLog
是否允许客户端可以查看可用模块列表，默认为可以
list = yes
#是否只读
read only = no
#忽略错误
ignore errors

2) View module rsync 124.71.8.140::

3) View the file rsync in the binding folder under the module 124.71.8.140::mag_sync

4) Pull the bound folder under the module to the target folder rsync -av 124.71.8.140::mag_sync /root/data/mongologs/

You can see that the file synchronization is complete

2. Password-protected mode

1) Modify the configuration file

#用户
uid = root 
#用户组
gid = root
#日志文件路径
log file = /var/log/rsyncd.log
secrets file = /etc/rsyncd/rsyncd.secrets
#模块名
[mag_sync]
#源文件路径
path = /usr/local/mongodb/data/auditLog
是否允许客户端可以查看可用模块列表，默认为可以
list = yes
#是否只读
read only = no
#忽略错误
ignore errors
#认证用户
auth users =root

2) Add password: vi /etc/rsyncd/rsyncd.secrets

Username Password

3) Give permission

chmod 600 /etc/rsyncd/rsyncd.secrets

4) Test (password input is required when the target server is connected)

rsync [email protected]::mag_sync

3. The target server calls the source server to pull the file and automatically obtains the password configuration

1) Save the password on the target server

vi /etc/rsync.password

Zz@1qaz2wsx

2) Give permission:

chmod 600 /etc/rsync.password

3) Pull source files

远程调用：rsync --password-file=密码存放路径 用户名@源服务器ip::模块名 要保存在哪个文件夹路径下
rsync -av --password-file=/etc/rsync.password [email protected]::mag_sync /root/data/mongologs/

3. Realize file synchronization regularly

Requirement : Let the rsync client automatically synchronize data with the server

Ideas:

1. Write a script to pull source files. (rsync)

2. Open a scheduled task and execute the script regularly. (cron system scheduling process)

The crontab . This command reads instructions from the standard input device and stores them in the "crontab" file for later reading and execution.

1. Check whether crontab is installed, if not installed, please install it yourself

rpm -qa | grep crontab

#vixie-cron软件包是cron的主程序； 
#crontabs软件包是用来安装、卸装、或列举用来驱动 cron 守护进程的表格的程序。
yum install  vixie-cron
yum install  crontabs  (centos安装指令)

#安装完以后开启crontab服务
service crond start

service crond start //start service
service crond stop //close service
service crond restart //restart service
service crond reload //reload configuration

View crontab service status: service crond status

2. There are two ways to add new scheduling tasks:

First create a new script file:

vi /root/data/pulllog.sh
rsync -av --password-file=/etc/rsync.password [email protected]::mag_sync /root/data/mongologs/

默认创建的这个sh问件是没有执行权限的，修改权限
chmod 777 pulllog.sh

分  时 天 月 星期  以root用户身份来运行  执行的脚本路径
*   *  *  *  *     root              /root/data/pulllog.sh

1) Enter on the command line: crontab -e and then add the corresponding task, wq save and exit.

2), Directly edit the /etc/crontab file, namely vi /etc/crontab, and add corresponding tasks.

crontab -e configuration is for a certain user, and editing /etc/crontab is for system tasks
View scheduling tasks
crontab -l //List all current scheduling tasks
crontab -l -u jp //List user jp All scheduled tasks

crontab -r //Delete all task scheduling jobs

3) View scheduling tasks

crontab -l

If not, restart the service

service crond restart

If there is an error, Linux will send an email to you during execution

cat /var/spool/mail/root

At this point, the file is executed regularly to pull the source file and synchronize it to the new folder.