Common Problems encountered during synchronization Rsync

---

A, Rsync service introduction

Rsync belong to a software tool for full volume and incremental synchronization of data for unix / linux / windows and many other operating system platforms.
Rsync software enables local replication, remote replication, or remote replication daemon mode. It is known for its delta-transfer algorithm, reducing the number of data sent over the network by transmitting only the difference information between the source and target files, thereby to achieve an incremental synchronization of data replication.

Two, Rsync works

1. Local data backup
2. Remote data transmission mode
3. Daemon mode data transmission

With rsync daemon way to achieve mainly through man rsync help to see usage.

#拉取数据的命令参数用法
Pull：rsync [OPTION...] [USER@]HOST::SRC... [DEST]
      rsync [OPTION...] rsync://[USER@]HOST[:PORT]/SRC... [DEST]
参数解释：
      [USER@]HOST::       --- 指定远程连接的认证用户
      SRC                 --- 指定相应的模块信息
      [DEST]              --- 将远程数据保存到本地的路径信息

#推送数据的命令参数用法
Push: rsync [OPTION...] SRC... [USER@]HOST::DEST
      rsync [OPTION...] SRC... rsync://[USER@]HOST[:PORT]/DEST
参数解释：
      [USER@]HOST::       --- 指定远程连接的认证用户
      SRC                 --- 指定本地要进行推送的数据信息
      [DEST]              --- 远程进行保存数据的模块信息

Third, the deployment process daemon

3.1 server deployment

1) Check the host environment

[root@backup ~]# cat /etc/redhat-release 
CentOS release 6.8 (Final)
[root@backup ~]# uname -r
2.6.32-642.el6.x86_64
[root@backup ~]# uname -m
x86_64

2) Check the software is installed

[root@backup ~]# rpm -qa rsync
rsync-3.0.6-12.el6.x86_64

3) edit the configuration file
the default configuration file does not exist, you need to create rsyncd.conf in the / etc directory.
Parameters Usage: man rsyncd.conf.

#created by user at 2018
###rsyncd.conf start###
uid = www                                 #管理备份目录的用户（需要在linux系统下创建虚拟用户）
gid = www                                 #管理备份目录的用户组
use chroot = no                           #保护模式，一般内网使用，可以选择禁用no
max connections = 200                     #可以同时处理的并发连接数
timeout = 300                             #连接超时，在指定时间内没有数据传输，自动释放连接
pid file = /var/run/rsyncd.pid
lock file = /var/run/rsync.lock
log file = /var/log/rsyncd.log    
ignore errors                             #备份传输过程中，忽略一些I/O产生的传输错误
read only = false                         #对备份目录开启读写权限，默认为true，只读模式
list = false                              #禁止通过客户端查询服务端配置的模块信息
hosts allow = 172.16.1.0/24               #允许指定网段或主机来访问备份目录
auth users = rsync_backup                 #认证用户（虚拟认证用户），不是真实存在的，非linux系统管理的虚拟用户
secrets file = /etc/rsync.pass            #设置免交互方式用户认证文件
[data]                                    #指定备份目录的模块名称
comment = "sersync dir by myself"
path = /data                              #以绝对路径方式，设置备份目录路径信息

4) Create a backup directory user management

[root@backup ~]# useradd www -M -s /sbin/nologin

5) Create a backup directory

[root@backup ~]# mkdir /data
[root@backup ~]# chown -R www.www /data

6) create a user authentication file

[root@backup ~]# echo 'rsync_backup:123456' >>/etc/rsync.pass
[root@backup ~]# chmod 600 /etc/rsync.pass

Additional content "User Authentication: password" to the authentication file, before switching to the additional English input method, and other end of the line without adding a space or tab content, preventing authentication error.
Vim editor can last line mode, enter: set list to check or use the command cat -A /etc/rsync.pass inspection.

7) Start Rsync service

[root@backup ~]# rsync --daemon
[root@backup ~]# netstat -lntp|grep rsync
tcp        0      0 0.0.0.0:873              0.0.0.0:*                LISTEN        12805/rsync         
tcp        0      0 :::873                   :::*                     LISTEN        12805/rsync
[root@backup ~]# echo '/usr/bin/rsync --daemon' >>/etc/rc.local

If the rsync service default port 873 is occupied by other services, use rsync --daemon --port = PORT start test command, more help see rsync --daemon --help.

3.2 Client Deployment

1) Check the host environment

[root@nfs ~]# cat /etc/redhat-release 
CentOS release 6.8 (Final)
[root@nfs ~]# uname -r
2.6.32-642.el6.x86_64
[root@nfs ~]# uname -m
x86_64

2) Check the software is installed

[root@nfs ~]# rpm -qa rsync
rsync-3.0.6-12.el6.x86_64

3) Create a certification file

[root@nfs ~]# echo '123456' >>/etc/rsync.pass
[root@nfs ~]# chmod 600 /etc/rsync.pass

Client certificate files only need to add a password, to authenticate the user to set the client through the rsync command.

4) data backup test

#客户端备份数据
[root@nfs ~]# rsync -avz /etc/hosts [email protected]::data --password-file=/etc/rsync.pass 
sending incremental file list
hosts

sent 180 bytes  received 27 bytes  414.00 bytes/sec
total size is 294  speedup is 1.42

#服务端检查备份数据
[root@backup ~]# ll /data/hosts
-rw-r--r-- 1 www www 294 Nov  6  2018 /data/hosts

Fourth, the wrong problem common Rsync backup summary

1. The client implementation of rsync command error

• 错误提示
  ERROR: The remote path must start with a module name not a /
• 演示过程
  [root@nfs ~]# rsync -avz /etc/hosts [email protected]::/data --password-file=/etc/rsync.pass
  ERROR: The remote path must start with a module name not a /
  rsync error: error starting client-server protocol (code 5) at main.c(1503) [sender=3.0.6]
• Problem solving
  rsync command syntax error, :: / data incorrect syntax, data for the module, not the directory, the correct wording :: data.

2. Configure-free backup data interactively, the client still need to enter the password problem

• 错误提示
  password file must not be other-accessible

• 演示过程
  [root@nfs ~]# rsync -avz /etc/hosts [email protected]::data --password-file=/etc/rsync.pass
  password file must not be other-accessible
  continuing without password file
  Password:
  sending incremental file list

  sent 26 bytes received 8 bytes 9.71 bytes/sec
  total size is 294 speedup is 8.65

• Problem solving
  Rsync client authentication file is not 600 permissions.

3. Rsync server firewall issues

• Error message
  No route to host
• 演示过程
  [root@nfs ~]# rsync -avz /etc/hosts [email protected]::data --password-file=/etc/rsync.pass
  rsync: failed to connect to 172.16.1.41: No route to host (113)
  rsync error: error in socket IO (code 10) at clientserver.c(124) [sender=3.0.6]
• Problem solving
  Rsync service is enabled with the firewall function, close it.

4. Rsync user authentication failures

• Error message
  ERROR: auth failed on module data
• 演示过程
  [root@nfs ~]# rsync -avz /etc/hosts [email protected]::data --password-file=/etc/rsync.pass
  @ERROR: auth failed on module data
  rsync error: error starting client-server protocol (code 5) at main.c(1503) [sender=3.0.6]
• Problem Solving
  1. The client rsync command to authenticate the user did not write right, or authentication password file is located at the end of the line there is an extra space or tab (cat -A /etc/rsync.pass check)
  2. client authentication file / etc / rsync .pass configuration error, no rsync_backup authenticate the user only needs to set authentication password.
  3. The server certification file rsync_backup: 123456 Available line or tab, note.
  4. The server certification file permissions are not /etc/rsync.pass 600.
  5. rsyncd.conf server configuration file, the file name of the certificate authentication and file name secrets file system parameter settings created inconsistencies.
  6. rsyncd.conf server configuration files, user management and backup catalog management user uid and gid set inconsistent.

5. Rsync service permissions prevent problems

• Error message
  Permission denied

• 演示过程
  [root@nfs ~]# rsync -avz /etc/hosts [email protected]::data --password-file=/etc/rsync.pass
  sending incremental file list
  hosts
  rsync: mkstemp ".hosts.dGfSQ1" (in data) failed: Permission denied (13)

  sent 180 bytes received 27 bytes 414.00 bytes/sec
  total size is 294 speedup is 1.42
  rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1039) [sender=3.0.6]

• Problem Solving
  1. /etc/rsyncd.conf file server, owner and managing user and the backup directory uid and gid set is a group inconsistent.
  2. backup directory file permissions are not 755.

6. Rsync backup directory service is abnormal

• Error message
  ERROR: chdir failed
• 演示过程
  [root@nfs ~]# rsync -avz /etc/hosts [email protected]::data --password-file=/etc/rsync.pass
  @ERROR: chdir failed
  rsync error: error starting client-server protocol (code 5) at main.c(1503) [sender=3.0.6]
• Problem-solving
  server /etc/rsyncd.conf file, backup directory path settings module path problems, check the directory exists, or path path correctly filled.

7. Rsync backup service module is abnormal

• Error message
  ERROR: Unknown module 'data'
• 演示过程
  [root@nfs ~]# rsync -avz /etc/hosts [email protected]::data --password-file=/etc/rsync.pass
  @ERROR: Unknown module 'data'
  rsync error: error starting client-server protocol (code 5) at main.c(1503) [sender=3.0.6]
• Problem Solving
  1. Client rsync command is executed, the specified module name is incorrect.
  2. The server /etc/rsyncd.conf file, the module name specified backup directory is incorrect.

Invalid user problems 8.Rsync

• Error message
  ERROR: invalid uid mine
• 演示过程
  [root@nfs ~]# rsync -avz /etc/hosts [email protected]::data --password-file=/etc/rsync.pass
  @ERROR: invalid uid mine
  rsync error: error starting client-server protocol (code 5) at main.c(1503) [sender=3.0.6]
• Problem-solving
  server management user virtual backup directory does not exist, can be created by "useradd user -M -s / sbin / nologin".

9. Rsync service is not open issue

• Error
  Connection refused
• 演示过程
  [root@nfs ~]# rsync -avz /etc/hosts [email protected]::data --password-file=/etc/rsync.pass
  rsync: failed to connect to 172.16.1.41: Connection refused (111)
  rsync error: error in socket IO (code 10) at clientserver.c(124) [sender=3.0.6]
• Problem Solution
  1. Check service: PS -ef | grep [r] Sync
  2. open the service: rsync --daemon

10. Rsync service connection timeout problem

• Error message
  Connection timed out
• 演示过程
  [root@nfs ~]# rsync -avz /etc/hosts [email protected]::data --password-file=/etc/rsync.pass
  rsync: failed to connect to 172.16.1.51: Connection timed out (110)
  rsync error: error in socket IO (code 10) at clientserver.c(124) [sender=3.0.6]
• Problem Solving
  Check the target host IP address, ping the target host to check whether the link is normal.
  [NFS the root @ ~] # of ping 172.16.1.41
  the PING 172.16.1.41 (172.16.1.41) 56 is (84) bytes of Data.
  64 bytes from 172.16.1.41: icmp_seq = 64. 1 TTL = Time = 0.438 MS
  64 bytes from 172.16. 1.41: icmp_seq = 2 ttl = 64 time = 0.543 ms

11. Rsync service slow connection problems

• Log Analysis
  [root @ Backup ~] # tailf /var/log/rsyncd.log
  2019/06/27 12:49:35 [22251] params.c: the Parameter () - Ignoring Badly Formed Line in the Configuration File: the ignore errors
  - - No rsync client to resolve host host on the server, resulting in reverse resolve the domain name, so that the connection is slow.
  12:49:55 2019/06/27 [22251] Lookup failed for 172.16.1.31 name: failure in the Temporary name Resolution
  2019/06/27 12:49:55 [22251] Connect from UNKNOWN (172.16.1.31)
  2019/06 / 27 12:49:55 [22251] Data to the rsync / from rsync_backup Unknown @ (172.16.1.31)
  2019/06/27 12:49:55 [22251] Receiving File List
  2019/06/27 12:49:56 [ 22251] sent 76 bytes received 81 bytes total size 294
• Problem solving
  [root @ Backup ~] # echo -e '172.16.1.31 \ TNFs' >> / etc / hosts
  [root @ Backup ~] # tailf /var/log/rsyncd.log
  2019/06/27 12:55: 06 [22268] params.c: the Parameter () - Ignoring Badly Formed Line in the Configuration File: the ignore errors
  --- can view the nfs server to host a proper reverse resolution process.
  12:55:06 2019/06/27 [22268] Connect from NFS (172.16.1.31)
  2019/06/27 12:55:06 [22268] Data to the rsync / from rsync_backup @ NFS (172.16.1.31)
  2019/06 / 27 12:55:06 [22268] Receiving File List
  2019/06/27 12:55:06 [22268] Sent Received 81 bytes Total 76 bytes size 294