Real-time data synchronization using sersync

The birth process of sersync

Sersync author: Former Kingsoft Company 周洋(Huajiao Live)
real-time data synchronization (inotify+rsync script) —> Develop real-time synchronization program (inotify+rsync)

Principle: https://www.bilibili.com/video/BV1TK411B7hU

Code repository: https://github.com/omaidb/sersync3
https://code.google.com/archive/p/sersync/source/default/source

Project address: https://github.com/omaidb/sersync3
https://github.com/wsgzao/sersync

Project download address: https://code.google.com/archive/p/sersync/downloads

Reference:
https://blog.csdn.net/m0_46095955/article/details/104032057
https://linux.cn/article-6032-1.html

sersyncIt is c++written in use and filters temporary files and repeated file operations generated by the Linux system file system (see the appendix for details, this filtering script is not implemented), so when combined with rsync synchronization, it saves runtime consumption and network resources. . Therefore it is faster.
sersyncThe configuration is very simple. The binary file and xml configuration file have been statically compiled and can be used directly.
sersyncUsing multi-threading for synchronization, especially when synchronizing larger files, can ensure that multiple servers remain synchronized in real time.
sersyncThere is an error handling mechanism to resynchronize the files with errors through the failure queue. If it still fails, the files that failed to synchronize will be resynchronized according to the set time.
sersyncNot only is it possible 实时同步, but it also comes with it crontab功能. You only need to turn it on in the xml configuration file, and you can synchronize the entire system at intervals as needed. No additional configuration of crontab function is required.
sersyncThere are socketextensions http插件to meet the needs of secondary development.

Deployment premise

Installedinotify-tools
rsync is 守护进程模式or configuredssh免密

# 安装inotify-tools
yum install -y inotify-tools

# 安装rsync
yum install -y rsync

Configure rsync server

https://blog.csdn.net/omaidb/article/details/121746997

Deploy sersync

一般情况下deployed rsyncinclient端

Download software binary package

https://github.com/omaidb/qiaofei_notes/tree/main/shell_code/sersync

https://github.com/wsgzao/sersync
sersync2.5.4 is the final version of the software.

# 下载sersync二进制包
wget -c https://raw.githubusercontent.com/wsgzao/sersync/master/sersync2.5.4_64bit_binary_stable_final.tar.gz

Unzip the software and save it to the corresponding directory

# 创建目录
mkdir -p /usr/local/sersync

# 解压文件
tar xf sersync2.5.4_64bit_binary_stable_final.tar.gz

# 查看解压的目录结构
tree GNU-Linux-x86

# 将sersync2修改为sersync
mv GNU-Linux-x86/sersync2 GNU-Linux-x86/sersync

# 将目录下文件移动到/usr/local/sersync/
mv GNU-Linux-x86/* /usr/local/sersync/

Configure the path variable of sersync

vim /etc/profile.d/sersync.sh

# 指定sersync程序的路径
export PATH="$PATH:/usr/local/sersync"

Make the path variable effective
source /etc/profile.d/sersync.sh

Modify sersync configuration file

vim /usr/local/sersync/confxml.xml

With 详细注释: https://github.com/omaidb/qiaofei_notes/tree/main/config_bak/sersync配置文件 2 configuration files

rsyncdDaemon mode configuration:rsyncd_confxml.xml
ssh免密Mode configuration:ssh_key_confxml.xml

sersync common parameters

# 查看 sersync 参数
sersync -h

parameter	explain
-d	Enable daemon mode
-r	Before monitoring, use the rsync command to push the monitoring directory and the remote host.
-n	Specify the number of daemon threads to start, the default is 10
-o	Specify `配置文件`, use `confxml.xml`file by default
`-m refreshCDN`	Enable refresh CDN module-plugin
`-m socket`	Enable socket module-plugin
`-m http`	Enable http module-plugin
`不加`-m parameter	The synchronization program is executed by default

# 后台启动sersync同步服务程序
## -d 启用守护进程模式
## -r 在监控前，将监控目录与远程主机用rsync命令推送一遍
## -o 指定配置文件
sersync -dro /usr/local/sersync/confxml.xml

Use service files to implement automatic startup at boot

vim /etc/systemd/system/sersync.service

# /etc/systemd/system/sersync.service
[Unit]
Description= sersync数据实时同步服务
# 当指定的配置文件存在时，服务才会启动
ConditionPathExists=/usr/local/sersync/confxml.xml

[Service]
# 启动命令，不要加-d
ExecStart=/usr/local/sersync/sersync -ro /usr/local/sersync/confxml.xml
# 服务停止时的命令
## 命令要以绝对路径执行
ExecStop=/usr/bin/killall sersync

[Install]
WantedBy=multi-user.target

After saving,重载服务

# 重载服务
systemctl daemon-reload

# 设置开机自启动
systemctl enable --now sersync.service

Real-time synchronization service debugging

Modify 配置文件parameters and proceed调试
vim /usr/local/sersync/confxml.xml

<!-- 开启debug模式 -->
<debug start="true"/>

debug模式After modification重启服务

# 重启服务
systemctl restart sersync

# journalctl实时查看日志
journalctl -u sersync -f

前台运行View using详细日志

# 停止服务
systemctl stop sersync

# 不加-d以前台模式打印日志输出
sersync -ro /usr/local/sersync/confxml.xml