用法示例：rsync -zavP --delete --exclude=.svn --exclude=.git --exclude=.gitignore --exclude=*.log --exclude=.gitattributes --exclude=storage /data/www/vhosts/fotophireapi.wondershare.com/ [email protected]::apache/data/www/vhosts/fotophireapi.wondershare.com/
1. Introduction
rsync is a commonly used Linux application for file synchronization.
It can synchronize files between a local computer and a remote computer, or between two local directories (but does not support synchronization between two remote computers). It can also be used as a file copy tool, substitution cpand mvcommand.
Its name rrefers to remote, and rsync actually means "remote sync" (remote sync). Unlike other file transfer tools (such as FTP or scp), the biggest feature of rsync is that it checks the existing files of the sender and receiver, and only transfers the changed parts (the default rule is that the file size or modification time changes).
Two, installation
If rsync is not installed on the local or remote computer, you can install it with the following command.
# Debian
$ sudo apt-get install rsync
# Red Hat
$ sudo yum install rsync
# Arch Linux
$ sudo pacman -S rsync
Note that rsync must be installed on both sides of the transmission.
Three, basic usage
3.1 -r Parameters
When the machine with rsync, as cpand mvalternative commands, the directory synchronization source to the destination directory.
$ rsync -r source destination
In the above command, it -rmeans recursion, that is, including subdirectories. Note that it -ris necessary, otherwise rsync will not run successfully. sourceThe directory represents the source directory and the destinationtarget directory.
If there are multiple files or directories that need to be synchronized, they can be written as follows.
$ rsync -r source1 source2 destination
The above command, source1, source2it will be synchronized to the destinationdirectory.
3.2 -a Parameters
-aParameters can be substituted -r. In addition to recursive synchronization, meta information (such as modification time, permissions, etc.) can also be synchronized. Since rsync uses file size and modification time by default to determine whether a file needs to be updated, -aratio is -rmore useful. The following usage is the common way of writing.
$ rsync -a source destination
destinationIf the target directory does not exist, rsync will automatically create it. After executing the above command, the source directory sourceis completely copied to the target directory destination, which forms the destination/sourcedirectory structure.
If you only want to synchronize sourcethe contents of the source directory to the target directory destination, you need to add a slash after the source directory.
$ rsync -a source/ destination
After the above command is executed, the sourcecontents of the directory will be copied to the destinationdirectory, and destinationa sourcesubdirectory will not be created below .
3.3 -n Parameters
If you are not sure what result will be produced after rsync is executed, you can use -nor --dry-runparameter to simulate the result of execution.
$ rsync -anv source/ destination
In the above command, the -nparameter simulates the result of the command execution and does not actually execute the command. -vThe parameter is to output the result to the terminal so that you can see what content will be synchronized.
3.4 --delete Parameters
By default, rsync only ensures that all the contents of the source directory (except for explicitly excluded files) are copied to the target directory. It does not keep the two directories the same, and it does not delete files. If you want to make the target directory a mirror copy of the source directory, you must use a --deleteparameter, which will delete files that only exist in the target directory but not in the source directory.
$ rsync -av --delete source/ destination
The above command, --deleteparameters that destinationbecome sourcea mirror image.
Four, exclude files
4.1 --exclude Parameters
Sometimes, we want to exclude certain files or directories during synchronization. At this time, we can use --excludeparameters to specify the exclusion mode.
$ rsync -av --exclude='*.txt' source/ destination
# 或者
$ rsync -av --exclude '*.txt' source/ destination
The above command excludes all TXT files.
Note that rsync will synchronize hidden files starting with "dot". If you want to exclude hidden files, you can write like this --exclude=".*".
If you want to exclude all files in a directory, but do not want to exclude the directory itself, you can write it as follows.
$ rsync -av --exclude 'dir1/*' source/ destination
Multiple exclusion modes can use multiple --excludeparameters.
$ rsync -av --exclude 'file1.txt' --exclude 'dir1/*' source/ destination
Multiple exclusion modes can also take advantage of the extended function of Bash's large extension, using only one --excludeparameter.
$ rsync -av --exclude={'file1.txt','dir1/*'} source/ destination
If there are many excluded patterns, you can write them into a file with one line per pattern, and then --exclude-fromspecify this file with parameters.
$ rsync -av --exclude-from='exclude-file.txt' source/ destination
4.2 --include Parameters
--includeThe parameter is used to specify the file mode that must be synchronized, and is often --excludeused in combination with it.
$ rsync -av --include="*.txt" --exclude='*' source/ destination
The above command specifies that all files will be excluded during synchronization, but TXT files will be included.
Five, remote synchronization
5.1 SSH protocol
In addition to supporting synchronization between two local directories, rsync also supports remote synchronization. It can synchronize local content to a remote server.
$ rsync -av source/ username@remote_host:destination
You can also synchronize remote content to the local.
$ rsync -av username@remote_host:source/ destination
By default, rsync uses SSH for remote login and data transmission.
Since early rsync did not use the SSH protocol, it was necessary to -especify the protocol with parameters, which was changed later. Therefore, the following -e sshcan be omitted.
$ rsync -av -e ssh source/ user@remote_host:/destination
However, if the ssh command has additional parameters, you must use the -eparameters to specify the SSH command to be executed.
$ rsync -av -e 'ssh -p 2234' source/ user@remote_host:/destination
In the above command, the -eparameter specifies that SSH uses port 2234.
5.2 rsync protocol
In addition to using SSH, if the rsync daemon is installed and running on another server, the rsync://protocol (default port 873) can also be used for transmission. The specific wording is to use a double colon to separate the server and the target directory ::.
$ rsync -av source/ 192.168.122.32::module/destination
Note that the above address is modulenot the actual path name, but a resource name specified by the rsync daemon, which is assigned by the administrator.
If you want to know the list of all modules allocated by the rsync daemon, you can execute the following command.
$ rsync rsync://192.168.122.32
In addition to using double colons in the rsync protocol, you can also directly use the rsync://protocol to specify the address.
$ rsync -av source/ rsync://192.168.122.32/module/destination
Six, incremental backup
The biggest feature of rsync is that it can complete incremental backups, that is, by default, only files that have changed are copied.
In addition to the direct comparison between the source directory and the target directory, rsync also supports the use of a reference directory, that is, the part that changes between the source directory and the reference directory is synchronized to the target directory.
The specific method is that the first synchronization is a full backup, and all files are synchronized in the base directory. Each subsequent synchronization is an incremental backup, only the part that has changed between the source directory and the base directory is synchronized, and this part is saved in a new target directory. This new target directory also contains all files, but in fact, only those changed files exist in this directory, and the other files that have not changed are hard links to the reference directory files.
--link-destThe parameter is used to specify the base directory during synchronization.
$ rsync -a --delete --link-dest /compare/path /source/path /target/path
In the above command, the --link-destparameter specifies the base directory /compare/path, and then the source directory /source/pathis compared with the base directory to find out the changed files and copy them to the target directory /target/path. Those files that have not changed will generate hard links. The first backup of this command is a full backup, and all subsequent backups are incremental.
Below is an example of a script that backs up the user's home directory.
#!/bin/bash
# A script to perform incremental backups using rsync
set -o errexit
set -o nounset
set -o pipefail
readonly SOURCE_DIR="${HOME}"
readonly BACKUP_DIR="/mnt/data/backups"
readonly DATETIME="$(date '+%Y-%m-%d_%H:%M:%S')"
readonly BACKUP_PATH="${BACKUP_DIR}/${DATETIME}"
readonly LATEST_LINK="${BACKUP_DIR}/latest"
mkdir -p "${BACKUP_DIR}"
rsync -av --delete \
"${SOURCE_DIR}/" \
--link-dest "${LATEST_LINK}" \
--exclude=".cache" \
"${BACKUP_PATH}"
rm -rf "${LATEST_LINK}"
ln -s "${BACKUP_PATH}" "${LATEST_LINK}"
In the above script, each synchronization will generate a new directory ${BACKUP_DIR}/${DATETIME}and ${BACKUP_DIR}/latestpoint the soft link to this directory. The next time it is backed up, it will be ${BACKUP_DIR}/latestused as the base directory to generate a new backup directory. Finally, ${BACKUP_DIR}/latestpoint the soft link to the new backup directory.
Seven, configuration items
| Options | Description |
|---|---|
| -a, ––archive | Archive mode, which means to transfer files recursively and keep all file attributes, which is equivalent to -rlptgoD (note that -H is not included) |
| -r, ––recursive | Process subdirectories in recursive mode |
| -l, ––links | Keep symbolic link files |
| -H, ––hard-links | Keep hard-linked files |
| -p, ––perms | Keep file permissions |
| -t, ––times | Keep file time information |
| -g, ––group | Keep file group information |
| -o, ––owner | Keep file owner information (super-user only) |
| -D | Keep device files and special files (super-user only) |
| -z, ––compress | Compress when transferring files |
| ––exclude=PATTERN | Specify to exclude a file matching pattern that does not need to be transferred |
| ––exclude-from=FILE | Read exclusion rules from FILE |
| ––include=PATTERN | Specify the file matching mode to be transferred |
| ––include-from=FILE | Read include rules from FILE |
| ––copy-unsafe-links | Copy link files that point to the SRC path outside the directory tree |
| ––safe-links | Ignore linked files that point to the SRC path outside the directory tree (default) |
| ––existing | Only update those files that already exist on the receiving end, and do not back up those newly created files |
| ––ignore-existing | Ignore those files that already exist on the receiving end, and only back up those newly created files |
| -b, ––backup | When there is a change, back up the old version of the file in the target directory |
| ––Backup-dir = DIR | Used in conjunction with -b, save the backup file to the DIR directory |
| ––Link-dest = DIR | Create a hard link file based on DIR when the file has not changed |
| ––delete | Delete files that are still on the receiving end but not on the sending end |
| ––delete-before | The receiver deletes before transmission (default) |
| ––delete-during | Recipient deletes during transmission |
| ––delete-after | Recipient deletes after transmission |
| ––delete-excluded | Delete excluded files at the same time on the recipient |
| -e, ––rsh=COMMAND | Specify a shell program that replaces rsh |
| ––ignore-errors | Delete even if I/O error occurs |
| ––partial | Keep those files that are not completely transferred for some reason, so as to speed up the subsequent re-transfer |
| ––progress | Display the transfer process during transfer |
| -P | Equivalent to ––partial ––progress |
| ––delay-updates | Save the file being updated to a temporary directory (the default is ".~tmp~"), and then update the target file after the transfer is completed |
| -v, ––verbose | Verbose output mode |
| -q, ––quiet | Reduced output mode |
| -h, ––human-readable | Use readable units for the output file size (eg, K, M, etc.) |
| -n, ––dry-run | Show which files will be transferred |
| ––list-only | Only list files without copying |
| ––rsyncpath=PROGRAM | Specify the path of the rsync command on the remote server |
| ––password-file=FILE | Read the password from FILE to avoid entering the password on the terminal, usually used when connecting to the rsync server in cron |
| -4, ––ipv4 | Use IPv4 |
| -6, ––ipv6 | Use IPv6 |
| ––version | Print version information |
| ––help | Display help information |
-
If you run the rsync command as a normal user, the owner of the synchronized file will be changed to this normal user.
-
若使用超级用户身份运行 rsync 命令，同步后的文件的属主将保持原来的用户身份。
八、参考链接
- How To Use Rsync to Sync Local and Remote Directories on a VPS, Justin Ellingwood
- Mirror Your Web Site With rsync, Falko Timme
- Examples on how to use Rsync, Egidio Docile
- How to create incremental backups using rsync on Linux, Egidio Docile