rsync service and inotfy real-time synchronization

rsync is a tool for file synchronization and backup. rsync is the next remote data synchronization tool for linux. It can efficiently transfer and update files between local or remote systems. Its main feature is to transmit only the changed parts to minimize the transmission volume and time.

rsync can synchronize files and directories between different hosts and even within the local file system. It uses a difference algorithm, called the rsync algorithm, to determine the differences between the source and destination files and then transfers only the differences.

When rsync is run for the first time, it copies the source file to the destination, and subsequent synchronization operations only transfer the changed parts. rsync is also able to compress data during transfer, further reducing the size of the transfer.

rsync supports many advanced features, such as preservation of file permissions, ownership, and timestamps, as well as the ability to backup and restore hard links, symbolic links, and device files. It can also sync remotely via secure protocols like SSH.

In summary, rsync is a powerful and flexible tool for synchronizing and backing up files quickly and efficiently, whether between local or remote systems.

It can quickly synchronize files and directories between multiple hosts through LAN/WAN, and appropriately uses the rsync algorithm to reduce data transmission. It
will compare different parts of two files and transmit the difference, so the transmission speed is quite fast.
rsync can copy, display directory attributes, copy files, and optionally compress and recursively copy.

2. Characteristics

rsync has a variety of features that make it a powerful and popular file synchronization and backup tool. Here are some of the main features of rsync:

Efficient differential transfer: rsync uses a differential algorithm to identify differences between source and destination files and transfers only the changed parts. This differential transmission method greatly reduces the amount and time of data transmission, and is especially suitable for synchronization of large files and large amounts of data.
Compressed transmission: rsync is able to compress data during transmission, thereby reducing the size of the transmission. This is particularly useful for remote synchronization over low-bandwidth connections.
Supports local and remote synchronization: rsync can synchronize files and directories on the local file system, as well as remotely between different hosts. It supports synchronization using local paths, SSH protocol, or other remote access methods.
Integrity check: rsync uses a checksum algorithm to ensure that transferred files are complete between the source and destination to avoid data corruption or loss.
Preserve file attributes: rsync can preserve various attributes of the source file, including file permissions, ownership, timestamps, etc. This is important to ensure that the synchronized files are consistent with the source files.
Partial file updates: If the synchronization process is interrupted, rsync is able to resume and continue synchronization from where it left off without retransmitting the entire file.
Multiple synchronization modes: rsync provides multiple synchronization modes, such as copying only newly added files, deleting files that do not exist in the target, keeping both sides in sync, etc. This makes rsync very flexible and can be customized according to needs.
Simulation mode: rsync has a simulation mode that allows you to preview the results of synchronization operations without actually performing the synchronization. This is useful for debugging and verifying synchronization operations.

Overall, rsync is efficient, flexible, and reliable, making it a popular file synchronization and backup tool.

3. Application scenarios

rsync is useful in many different application scenarios. Here are some common ones:

File backup and synchronization: The most common use of rsync is to back up and synchronize files or directories. It ensures that files remain consistent between the source and target, and only the changed parts are transferred during incremental updates. This is useful for regularly backing up important data, syncing files to a remote server, or creating file versioning.
Remote server synchronization: rsync can achieve secure remote file synchronization through SSH protocol. This is useful when you need to synchronize files from your local computer to a remote server (such as website file synchronization) or when synchronizing files between multiple servers.
Distribute and copy large files: rsync's differential transfer and compression capabilities make it very efficient when you need to distribute or copy large files (such as software updates, media files, etc.) to multiple destinations. It only transfers the changed parts and reduces the amount of data transferred through compression.
Incremental backup: rsync is suitable for incremental backup, that is, only the changed files or directories in the source are backed up, rather than the entire backup. This saves storage space and transfer time, and provides an efficient recovery point.
File migration: rsync can migrate files and directories between different file systems, hard disks, or servers. It ensures the integrity of file attributes and content and reduces transfer time during migration.
High-availability configuration: rsync can be used in conjunction with other tools, such as file system monitoring or load balancers, to achieve high-availability configurations. For example, rsync is used in a clustered environment to synchronize files periodically to maintain consistency across multiple nodes.
Data synchronization and replication: rsync can be used to synchronize and replicate data within a data center or distributed system. It can help ensure data consistency across multiple storage locations and can handle large amounts of data.

Overall, rsync's flexibility makes it useful in many different application scenarios, whether it is file backup, synchronization, migration, or high-availability configuration and data replication.

4. Data synchronization method of rsyns

Data backup: pull (download) pull

Data recovery: push (upload) push

5. rsync transmission mode

1. Local transmission

rsync local transfer mode is a mode for file transfer and synchronization on the local file system. It is suitable for scenarios where files need to be synchronized or copied on the same computer. The following is an example command for local transfer using rsync:

Sync directory:

rsync -av /path/to/source /path/to/destination

This will recursively synchronize all files and subdirectories in the source directory to the target directory. Whenever this command is run, rsync detects differences between the source and destination and makes incremental updates.

Only copy new files:

rsync -av --ignore-existing /path/to/source /path/to/destination

This command copies files in the source directory that do not already exist in the target directory to the target directory. Files that already exist in the target directory will be ignored and not copied.

Sync folder:

rsync -av /path/to/source/ /path/to/destination/

This command will synchronize all files and subdirectories in the source directory and keep the contents of the target directory consistent with the source directory. The trailing slash "/" means to copy the contents of the directory without including the directory itself.

Delete non-existent files:

rsync -av --delete /path/to/source /path/to/destination

In this command, if a file exists in the target directory that does not exist in the source directory, rsync will delete it to maintain synchronization between the source and target.

These commands demonstrate basic usage of rsync for file transfer and synchronization in a local environment. You can adjust the command parameters and options according to your specific needs. Remember to double-check the command before performing any file operations and confirm that the operation is risk-free.

2. Remote transmission

The rsync remote transfer mode is a mode for file transfer and synchronization between local and remote systems through the SSH protocol. It can safely copy files between two computers and maintain file consistency. The following is an example command for remote transfer using rsync:

Copy from local to remote system:

rsync -av -e ssh /path/to/source user@remote:/path/to/destination

This command will synchronize files and directories in the local source directory to the remote target directory. -e ssh Options specify using the SSH protocol for connection and transfer.

Copy from remote system to local:

rsync -av -e ssh user@remote:/path/to/source /path/to/destination

This command will synchronize files and directories in the remote source directory to the local target directory.

Only copy new files:

rsync -av --ignore-existing -e ssh /path/to/source user@remote:/path/to/destination

This command will only copy files in the local source directory to the remote destination directory that do not already exist in the remote destination directory.

Sync folder:

rsync -av -e ssh /path/to/source/ user@remote:/path/to/destination/

This command will recursively synchronize all files and subdirectories in the local source directory to the remote target directory. The trailing slash "/" means to copy the contents of the directory without including the directory itself.

Delete non-existent files:

rsync -av --delete -e ssh /path/to/source user@remote:/path/to/destination

In this command, if files exist in the remote destination directory that do not exist in the local source directory, rsync will delete them to maintain synchronization between the source and destination.

These commands demonstrate basic usage of remote transfer and synchronization using rsync. You will need to replace the examples in the examples /path/to/source with your actual source directory paths, user@remote:/path/to/destination with the username, hostname, and destination directory path of the remote system. You can also adjust other parameters and options as needed. Please make sure that before performing any file operations, you double-check the command to confirm that the connection and transfer settings are correct and that the operation is risk-free.

3. Daemon process

The rsync daemon mode refers to starting rsync as a daemon on the remote server to receive and process rsync client requests over the network. This mode can provide higher transfer efficiency because the rsync server can maintain a persistent connection with the client, avoiding the overhead of re-establishing the connection for each transfer. The following are the general steps for starting the rsync daemon:

Configure rsync server:
- First, make sure rsync is installed on the remote server.
- Create a rsyncd.conf configuration file named , usually located in /etc/rsyncd.conf or /etc/rsync/rsyncd.conf.
- Specify information such as the directory to be shared and access permissions in the configuration file. For example:
```
uid = nobody
gid = nobody
[mydata]
path = /path/to/shared/directory
read only = yes
```
  In the above configuration, mydata it is the share name and /path/to/shared/directory the directory path to be shared.
- Configure other options, such as authentication method, log path, etc. Make settings as needed.
Start the rsync daemon:
- Start rsync in daemon mode using the following command:
```
rsync --daemon
```
- rsync will read the settings in the configuration file and start a daemon on the server.
Connect to rsync server:
- Use the rsync command on the client to connect to the rsync server and perform operations such as file transfer.
```
rsync -av /path/to/local/source rsync://server_address/mydata
```
- server_address is the IP address or hostname of the rsync server and mydata is the share name specified in the configuration file.

In this way, the rsync client can connect to the rsync server and perform file transfer and synchronization operations while maintaining a persistent connection.

Note that starting the rsync daemon requires careful handling of security and access control. You can find more details about configuration and security in rsync's official documentation.

6. Configuration environment

1. First configure the yum source

2. Turn off the firewall, graphical, mount, and set tolerance mode

3. Set IP

4. Restart and install httpd

systemctl restart network

yum install -y httpd

5. Install rsync

Generally it exists in Linux and does not need to be installed.

6. Building services

7. Configuration file environment

vim /etc/rsyncd.conf

Go to the configuration file

8. Create a user -s to specify that you cannot log in to the operating system.

useradd -s /sbin/nologin -M rsync

9. Create a synchronized directory and change the owner and group

10. Create a password file and grant 600 permissions

echo "jx_mpy:2305" > /etc/rsync.passwd

Write to passwd under etc

chmod 600 /etc/rsync.passwd
restart the service

systemcti restart rsyncd

After the server configuration is completed, enter netstat -anptl to verify whether it is started and then switch to the client.

11. Configure the client to be consistent with the server configuration environment

yum source

mount

permissive model

firewall

Graphical

Allocate ip

12. The file created by the client is user.txt

cat /etc/passwd > /user.txt

View cat /user.txt

So far the user file has been successfully created

13. Modify file content to achieve synchronization

systemctl start rsyncd

success

Return to server verification

cat user view

Switching back to the server

Enter vim /user.txt and change the file after entering.

Delete 40 lines, keep 3 lines and save

Execute again

Return to server and verify successfully

14. inotfy real-time synchronization

Install the inotfy package first

Install httpd

Open port 80 for verification

View the homepage file cd /var/www/html/

Also install httpd on the client

Restart httpd

Verify port number netstat -anptl

Enter vim /etc/rsyncd.conf in the server section to change the configuration file

Save, exit and restart rsyncd

Server side configuration completed

The client enters vim /etc/sysctl.conf

sysctl -p refresh configuration completed

15. Verification

Open a terminal

Enter inotifywait -mrq -e modify,create,move,delete /var/www/html in the home directory of the first terminal

He will enter a monitoring state

When creating, deleting or editing content, the listening content on the left will change independently.

The ~ inside is also a cache file

16. Script

Explanation: The second and third lines mean to monitor the directory

The fourth line means that if there is a file in the target directory, delete it first and then upload it to the corresponding module.

Upload the files in the client's mul directory to the server segment

The path corresponding to the module

while syntax (it is a loop syntax, indicating that as long as the event can be reached, do this thing, and the loop will end after the completion)

Execute the command before the pipe character after reading the directory event file

Do one thing to achieve synchronization and then end it

17. Script monitoring verification

Switch to the server to see if it is successful

Web page verification

Enter the IP address to query the results

Change the file content vim index.html under the server again, save and exit

Re-enter verification on the left

Switching back to server verification (successful verification of refreshed content)

Enter vim 1.sh on the server to edit

Enter the script

throw him backstage

The server deletes index.html

Return to client

Change vim index.html

Keep two hello worlds

Return to server

Check that the server displays correctly