In the work, you may encounter the problem of single-point Redis migrating data to Redis cluster, but you can't do it with troublesome operation and maintenance. In order to facilitate the research and development of self-migrating data, I wrote a simple Redis migration tool here, hoping to be useful to those in need.
This tool supports:
- Single-point Redis to single-point Redis migration
- Single point Redis to Redis cluster migration
- Redis cluster to Redis cluster migration
- Redis cluster to single-point Redis migration
The tool has been compiled into a multi-platform command, just download the binary file directly from Github and execute it.
Project address: https://github.com/icowan/redis-tool After pulling down the code, you can directly execute the command make to compile executable files for multiple platforms, and you need to rely on the golang compiler.
- Windows amd64: redis-tool-windows-amd64.exe
- MacOS amd64: redis-tool-darwin-amd64
- Linux amd64: redis-tool-linux-amd64
- Linux arm64: redis-tool-linux -arm64
View Usage:
$ chmod a+x redis-tool-linux-amd64
$ ./redis-tool-linux-amd64 -h
Supported data types
- string
- hash hash list
- list list
- sorted-set ordered set
how to use
Download the command and execute it after authorization. /redis-tool-linux-amd64 -h You can view all the functions supported by the tool:
$ ./redis-tool-darwin-amd64 migrate -h
数据迁移命令
Usage:
redis-tool migrate [command]
Examples:
支持命令:
[hash, set, sorted-set, list]
Available Commands:
all 迁移所有
hash 哈希列表迁移
list 列表迁移
set redis set 迁移
sorted-set 有序集合迁移
Flags:
-h, --help help for migrate
--source-auth string 源密码
--source-database int 源database
--source-hosts string 源redis地址, 多个ip用','隔开 (default "127.0.0.1:6379")
--source-prefix string 源redis前缀
--source-redis-cluster 源redis是否是集群
--target-auth string 目标密码
--target-database int 目标database
--target-hosts string 目标redis地址, 多个ip用','隔开 (default "127.0.0.1:6379")
--target-prefix string 目标redis前缀
--target-redis-cluster 目标redis是否是集群
Use "redis-tool migrate [command] --help" for more information about a command.
Parameter Description:
- --source-auth: source redis password, fill in if there is one
- --source-database: source database, default is 0
- --source-hosts: source redis address, multiple ips of the cluster are separated by',' (default "127.0.0.1:6379")
- --source-prefix: source redis prefix, optional
- --source-redis-cluster: Whether the source redis is a cluster, the default is false
- --target-auth: migration target redis password, fill in if there is one
- --target-database: migration target database, default is 0
- --target-hosts: Migration target redis address, multiple ips of the cluster are separated by',' (default "127.0.0.1:6379")
- --target-prefix: migration target redis prefix, optional
- --target-redis-cluster: Whether the migration target redis is a cluster, default false
Migrate the data of a single key
Here are two examples, the others are not too different.
Hash type
You can view the instructions through the command redis-tool migrate hash -h
$ redis-tool migrate hash helloworld \
--source-hosts 127.0.0.1:6379 \
--target-redis-cluster true \
--target-hosts 127.0.0.1:6379,127.0.0.1:7379 \
--target-auth 123456
Ordered set
You can use the command redis-tool migrate sorted-set -h to view the instructions
. The amount of data in the ordered set may be relatively large, so the cut is performed in units of 50000. I have tested the migration of nearly 170 million pieces of data, which took more than 40 minutes.
$ redis-tool migrate hash helloworld \
--source-hosts 127.0.0.1:6379 \
--target-redis-cluster true \
--target-hosts 127.0.0.1:6379,127.0.0.1:7379 \
--target-auth 123456
Migrate all key data to support wildcard filtering
You can view the instructions through the command redis-tool migrate all -h
$ redis-tool migrate all "ipdetect:*" \
--source-hosts 127.0.0.1:6379 \
--target-redis-cluster true \
--target-hosts 127.0.0.1:6379,127.0.0.1:7379 \
--target-auth 123456
This command will compile all the matched keys, and then migrate gradually according to the key type.
tail
A relatively simple tool written in golang, which is mainly used when Redis does not have persistence or when multiple sets of Redis are migrated to one set of Redis.
Hope it is useful to everyone, thank you!
Author: CreditEase Institute of Technology Cong