[Linux] remote data synchronization

rsync command Detailed

rsync command is a remote data synchronization tool through LAN / WAN quickly synchronize files between multiple hosts. rsync so-called "rsync algorithm" to make files between local and remote hosts two synchronized, the algorithm only transmit two different parts of the file, but not always the whole transmission, and therefore very fast. rsync is a very powerful tool, its command, there are many options Features, we have the following options one by one its analysis shows.

rsync properties

And you can update the entire directory tree and file systems;
Selectively retaining symbol chain, hard link file belongs, permissions, equipment, and time;
For installations, without any special privileges requirements;
For a plurality of files, the internal pipeline delay reducing waiting file;
Use rsh, ssh, or direct transfer port as an input port;
Support anonymous rsync to synchronize files, it is the ideal imaging tool;

First, install rsync

centOS:

yum -y install rsync

Ubuntu

apt-get -y install rsync

Verify that the installation was successful

Command: rsync --version # View rsync version

[root@localhost ~]# rsync --version
rsync  version 3.1.2  protocol version 31
Copyright (C) 1996-2015 by Andrew Tridgell, Wayne Davison, and others.
Web site: http://rsync.samba.org/
Capabilities:
    64-bit files, 64-bit inums, 64-bit timestamps, 64-bit long ints,
    socketpairs, hardlinks, symlinks, IPv6, batchfiles, inplace,
    append, ACLs, xattrs, iconv, symtimes, prealloc

rsync comes with ABSOLUTELY NO WARRANTY.  This is free software, and you
are welcome to redistribute it under certain conditions.  See the GNU
General Public Licence for details.

Two, rsync common options

1 == - v, --verbose verbose output ==
2 == - q, --quiet silent mode, no transmission, even if wrong, there is no information. ==
3 -c, --checksum open check switch, forced to verify the file transfer
4 == - a, --archive filing mode, transfer files recursively expressed, and maintaining all file attributes, equal -rlptgoD ==
5 == - r, --recursive subdirectories recursive processing mode ==
6 -R, --relative using relative path information
7 -b, --backup create a backup, that is, when the objective has been there for the same file name, the old file is renamed ~ filename. You can use --suffix option to specify a different backup file prefix.
8 --backup-dir backup file (e.g., ~ filename) stored in the directory.
9 -suffix = SUFFIX backup file prefix definitions
10 -u, --update only be updated, that is, to skip all already present in the DST, and the file later than files to back up. (Do not overwrite the updated file)
11 == - l, --links reservations soft link ==
12 == - L, --copy-links would like to be treated the same as regular file processing soft link ==
13 --copy-unsafe-links only copy link pointing to the path tree other than the SRC
14 --safe-links pointing to ignore the path of the directory tree outside SRC link
15 -H, --hard-links preserves hard links
16 == - p, --perms to keep file permissions ==
17 == - o, --owner owner information file holding ==
18 == - g, --group keep the file is a set of information ==
19 == - D, --devices holding device file information ==
20 == - t, --times keep the file time information ==
21 -S, --sparse for special handling sparse files to save space for DST
22 -n, --dry-run reality which files will be transferred
23 -W, --whole-file copy files, not the incremental detecting
24 -x, --one-file-system do not cross file system boundaries
25 -B, --block-size = SIZE test algorithm block size, the default is 700 bytes
26 -e, --rsh = COMMAND specify rsh, ssh way data synchronization
27 --rsync-path = PATH specify where the information rsync command on a remote server
28 -C, --cvs-exclude CVS method using the same and automatically ignore the file, the file to exclude undesired transfer
29 --existing only update files that already exist in the DST, rather than backing up the newly created files
== 30 --delete delete those DST SRC is not in the file (if the source file does not exist, it has not even think about the destination, delete it
) ==
31 == - delete-excluded specifically designated number to be deleted in the destination file. ==
32 --delete-after after the transfer is complete before removing
33 --ignore-errors in a timely manner IO error also deleted
34 --max-delete = NUM NUM deleted files up to
35 --partial keep those files for some reason is not completely transferred, so accelerate the subsequent re-transmission
36 --force forcibly remove directories, even if not empty
37 --numeric-ids does not match the user and group ID numbers for the user and group names
38 --timeout = TIME IP timeout in seconds
39 -I, --ignore-times not to skip those files have the same length of time and
When 40 --size-only when deciding whether to back up files, regardless of file size only look Documents
41 --modify-window = NUM stamp window determines whether the file is used in the same time, the default is 0
42 -T --temp-dir = DIR creates temporary files in DIR
43 --compare-dest = DIR DIR same comparison in the file to determine whether you need to back up
44 == - P equivalent to --partial, keep those documents for some reason not fully transfer to speed up subsequent retransmission ==
45 --progress display the backup process
46 == - z, --compress backup file compression processing in the transmission ==
47 --exclude = PATTERN exclude undesirable pattern specified file transfer
48 --include = PATTERN need not preclude the specified file transfer mode
49 --exclude-from = FILE exclude specified pattern file FILE
50 --include-from = FILE FILE, does not exclude that match the specified pattern
51 --version Print version information
52 --address bound to a specific address
53 --config = FILE specify a different configuration file, do not use the default file rsyncd.conf
54 == - port = PORT specify other rsync service port ==
55 --blocking-io to use blocking IO for remote shell
56 -stats transmission status gives some files
57 --progress reality transmission process in the transmission
58 --log-format = formAT specified log format file
59 == - password-file = FILE == password obtained from the FILE
60 --bwlimit = KBPS limit I / O bandwidth, KBytes per second
61 -h, --help display help information

Three, rsync Use Examples

Example 1: synchronize two directories on the local server

Synchronize two directories on the local machine, use the command == rsync -zvr ==

[root@localhost ~]# rsync -zvr /opt/seafile-data/ /mnt/
sending incremental file list
db/
db/aria_log.00000001
db/aria_log_control
db/ib_logfile0

The above command:

-z open compression
-v verbose print more information
-r recursive

After executing the above command, you will find rsync copy will affect the timestamp information file, then because the default rsync does not protect timestamp information

note:

/ Opt / seafile-data / writing of such synchronization seafile-data folder and all files excluding seafile-data folder

/ Opt / seafile-data written in this way the entire seafile-data sync folder, the folder comprising seafile-data

Example 2: In the sync, retention time stamp -a (achive mode: recursive mode, reserved symbols links, to retain the authority information, a time stamp, and the owner, group information)

[root@localhost ~]# rsync -avz /opt/seafile-data /mnt/
sending incremental file list
seafile-data/
seafile-data/db/
seafile-data/db/aria_log.00000001
seafile-data/db/aria_log_control
seafile-data/db/ib_logfile0

Note that this time you will find that the time stamp source, dest file, the information belongs to the main group and so is a constant

[root@localhost ~]# ls -l /opt/seafile-data/logs/seafile/seahub.log /mnt/seafile-data/logs/seafile/seahub.log
-rw-r--r-- 1 root root 15858 3月  21 18:23 /mnt/seafile-data/logs/seafile/seahub.log
-rw-r--r-- 1 root root 15858 3月  21 18:23 /opt/seafile-data/logs/seafile/seahub.log

Example Three: only sync a file

Just specify the file name in the rsync command can

[root@localhost ~]# rsync -v /opt/seafile-data/seafile/conf/install.sh /mnt/
install.sh

sent 19,858 bytes  received 35 bytes  39,786.00 bytes/sec
total size is 19,776  speedup is 0.99

ll /mnt/install.sh 
-rw-r--r-- 1 root root 19776 3月  25 14:05 /mnt/install.sh

Example 4: synchronized from the local to the remote server

[root@localhost ~]# rsync -avz /opt/seafile-data [email protected]:/mnt/
[email protected]'s password:  输入远端服务器密码
sending incremental file list
seafile-data/
seafile-data/db/
seafile-data/db/aria_log.00000001
seafile-data/db/aria_log_control

......
......
sent 1,809,853,100 bytes  received 10,547 bytes  28,501,789.72 bytes/sec
total size is 2,005,006,840  speedup is 1.11

When performing synchronized movements and remote server, you need to specify the username, ip. Also specify the destination directory on the remote server, the format is: username @ machineIP: Path

That format is:

== rsync -avz need to synchronize the local file absolute path to the remote user name @ distal IP: save remote files absolute path ==

This process, rsync will be asked to enter a password. But if you have a script to automatically run the backup action, you probably do not want to manually enter a password that

Example 5: synchronized from a remote server to the local

[root@localhost ~]# rsync -avz [email protected]:/mnt/seafile-data /mnt/
[email protected]'s password:  输入密码
receiving incremental file list
seafile-data/
seafile-data/db/
seafile-data/db/aria_log.00000001
......
......
sent 4,698 bytes  received 1,785,742,775 bytes  31,056,477.79 bytes/sec
total size is 2,005,006,840  speedup is 1.12