1. Introduction
1. Know that
Rsync (remote synchronize) is a remote data synchronization tool that can quickly synchronize files between multiple hosts through LAN/WAN. Rsync uses the so-called "Rsync algorithm" to synchronize files between two hosts, local and remote. This algorithm only transfers different parts of the two files, rather than the entire transfer each time, so it is quite fast.
Rsync supports large Most Unix-like systems, whether Linux, Solaris, or BSD are well-tested
In addition , it also has corresponding versions under the Windows platform, such as tools such as cwRsync and Sync2NAS
2. Principle
Rsync was originally a tool used to replace rcp. It is currently maintained by rsync.samba.org, so the format of the rsync.conf file is similar to the main configuration file of samba; Rsync can be used through rsh or ssh, or as a daemon Mode to run
When daemon mode, the Rsync server will open a port 873 and wait for the client to connect. When connecting, the Rsync server will check whether the password matches. If the password is checked, the file transfer can be started. When the first connection is completed, the entire file will be transferred once, and then only incremental backups will be required.
3. Features
1. It can mirror and save the entire directory tree and file system;
2. It is easy to maintain the original file permissions, time, soft and hard links, etc.;
3. It can be installed without special permissions;
4. Optimized process, High efficiency of file transmission;
5. You can use rsh, ssh and other methods to transfer files, of course, you can also use direct socket connection;
6. Support anonymous transmission
7. Similar to scp, cp ls rm command
4. It can be realized:
It is an excellent tool that can realize full and incremental local or full data synchronization backup of different platforms such as Windows, Linux, unix, etc.
Two.rsync parameters:
-H 保留源文件的硬链接文件
-r 递归模式，包含目录及子目录的所有信息
-z 在传输文件的同时进行压缩
-v 显示同步过程的信息
-t 保留文件的时间标记
-o 保留文件的属主标记
-p 保留了文件的权限标记
-D 保留了设备文件和一些特殊的文件
-S 对零散文件的处理
-g 保留了文件的属组信息
--version 查看rsync版本
Three.rsync features:
1. Supports copying common files and special files, such as: symbolic links, block devices, character devices, etc.
2. Supports the exclusion function during synchronization
3. Supports copying of all attributes such as maintaining permissions (such as cp -p)
4. Supports incremental synchronization , only the changed data is synchronized, and the transmission efficiency is high
Four. rsync basic functions:
Copy files locally:
rsync /etc/hosts /opt/
Copy the directory locally ( -r )
rsync -r /etc /opt/
Delete the files in the directory (need to create an empty directory of /null):
rsync -r --delete /null/ /tmp/
|
Null
explanation --- In fact, it is to synchronize an
empty directory
with a directory with files
In this way, the directory with files will also become an empty directory after synchronization, which is the principle of rsync deletion
|
rsync to copy files to remote server:
scp -rp /etc/hosts 10.0.0.4/tmp ##----需要输入远程服务器的密码
Copy the directory to the remote server:
[root@chenleilei ~]# rsync /etc/hosts 10.0.0.4:/tmp
[email protected]'s password:
Four. Some small functions of rsync
cp function:
复制目录:
rsync -r /etc /tmp
复制文件
rsync -a /etc/hosts /tmp
scp remote copy function:
rsync /etc/hosts 10.0.0.4:/tmp
ls function:
[root@chenleilei ~]# rsync chenleilei/123.txt
-rw-r--r-- 0 2018/05/03 21:35:51 123.txt
rm function (can only delete all files in the directory)
1.创建一个空目录 chenleilei
2.rsync --delete -r 123/ chen/
Description of this rm function: The effect of rm deletion is only due to the additional synchronization function of rsync, which
will actively synchronize the contents of one directory with another directory, so that the synchronized directory is consistent with the synchronized directory. If If the synchronized directory is empty,
the synchronized directory will also be empty. Of course, if there is a file that needs to be synchronized, the
synchronized content will be replaced by the actively synchronized directory after the synchronization is successful.
5. Backup method:
full backup
全量备份会对服务器进行整体备份,一般第一次启动rsync的时候会进行一次全量备份
Incremental backup
增量备份只会对新增或者修改的文件进行备份
centos6以后的增量备份原理是:检查源数据与目标数据的差异,检查到有差异的文件立即备份到目标服务器,也就是边比对,边同步
6. Improve security:
通过soket(进程方式) 传输文件和数据(服务端客户端) ----重点掌握
Use rcp, rsh, ssh and other methods to cooperate with tunnel encryption to transfer files (rsync does not encrypt data)
Support anonymous or authenticated (anonymous: no system user) process mode transmission, to achieve convenient and safe data backup
7. Enterprise work usage scenarios:
1. Use timed tasks + rsync for data synchronization [内部员工使用场景]
2. Use real-time tasks + rsync for data synchronization[外部远程使用场景]
8. Project plan: Common cluster architecture backup plan for production scenarios---全网备份方案
借助cron+rsync 把所有客户服务器数据同步到备份服务器
全网服务器数据备份解决方案提出和负责实施
1.针对中哟啊数据备份混乱,向领导提出全网数据的备份解决方案
2.通过本地打包备份,使用 rsync结合 inotify 服务 实现全往数据统一备份到一个固定的服务器
然后保存在存储服务器中,并使用脚本检查,并通知管理员备份结果.
3.定期将IDC机房的数据备份到公司的内部服务器,防止机房地震及火灾问题导致的数据丢失.
Nine. rsync synchronization method:
1.本地同步方式: rsync 选项 源 目标
例子: rsync -r /etc /opt/
2.shell远程同步方式:
3.常用同步参数 -rp :
-r : 递归处理
-p: 保留权限
[email protected]: 远程主机用户身份和IP地址
4.文件的拉取和推送操作:
4.1 拉取远程数据(pull): rsync -rp [email protected]:/tmp/hosts /etc/
4.2 推送到远程服务器(push): rsync -rp /etc/hosts [email protected]:/tmp
5.目录的拉取推送操作: (/ 和不加 / 的区别是 加/是把目录中内容推送, 不加/ 是推送目录)
5.1 推送本地目录到远程/tmp目录下 rsync -rp /etc/sysconfig [email protected]:/tmp
5.2 拉取远程目录到本地当前目录下 rsync -rp [email protected]:/tmp/chen .
Ten.rsync configuration file - configuration daemon (/etc/rsyncd.conf daemon configuration file):
rsync默认配置文件没有被创建,它默认的配置文件存放地: /etc/rsyncd.conf
| #rsync_config #created by HQ at 2017 ##rsyncd.conf start## uid = rsync ##User id gid = rsync ##User group gid use chroot = no ##Security configuration. It is best to close max connections = 200 ##Maximum number of connections timeout = 300 ##Timeout time pid file = /var/ run/rsyncd.pid ##Process pid when the service is running (you can kill it by kill after viewing the pid) lock file = /var/run/rsync.lock ##Process lock file (locked after exceeding the maximum connection) log file = / var/log/rsyncd.log ##Program running log file ignore errors ## Ignore errors (to avoid stopping after an error is reported, so temporarily ignore errors and continue backup) read only = false ##Whether read-only force read-write list = false ##list hosts allow=172.16.1.0/24 ##allow hosts deny=0.0.0.0/32 ##deny auth users=rsync_backup secrets file=/etc/ rsync.password [backup] comment = "backup dir by oldboy" path = /backup ##The directory where the module is saved |
#created by chen at2018
##rsyncd.conf start
uid = rsync
gid = rsync
use chroot = on
max connections = 200
timeout = 300
pid file = /var/run/rsyncd.pid
lock file = /var/run/rsync.lock
log file = /var/log/tsyncd.log
ignore errors
hosts allow = 172.16.1.0/24
hosts deny = 0.0.0.0/32
auth users = rsync_backup
secrets file = /etc/rsync.passwd
[backup]
comment = "backup dir by chenleilei"
path = /backup
写入到/etc/rsyncd.conf文件中
Eleven. Create user virtual user rsync
useradd -s /sbin/nologin -M rsync
12. Create a user password file (security authentication file)
echo ‘rsync_backup:123456‘ >/etc/rsync.passwd
chmod 600 /etc/tsync.passwd ----给这个权限是不想让除了root之外的其他用户看到
Thirteen. Create and create a backup directory /backup
mkdir -p /backup
chown rsync.rsync /backup
Basic configuration complete
14. Start the daemon
rsync --daemon
他有进程信息 通过 ps -ef | grep rsync 查看
Since then the esync service has been configured