迁移数据到Redis Cluster

其他 2019-04-17 01:30:36 阅读次数: 0
版权声明：本文为博主原创文章，未经博主允许不得转载。 https://blog.csdn.net/weixin_42018518/article/details/88815157

目录

一、数据迁移脚本树形图

二、数据丢失问题

三、执行脚本redis_data_migration.sh

四、全局参数配置文件redis_data_migration.conf

五、原项目对应Redis服务器IP&PORT文件

六、Redis Cluster IP&PORT文件

一、数据迁移脚本树形图

/home/work/wufei/migration

├── bin

│   └── redis_data_migration.sh

├── conf

│   └── redis_data_migration.conf

├── ip_port

│   ├── migration_redis_ip_port.bim

│   ├── migration_redis_ip_port.im

│   ├── redis_cluster_ip_port.bim

│   ├── redis_cluster_ip_port.im

│   └── redis_cluster_ip_port.inf

├── del

│   ├── migration_del.log

│   └── migration_del.success

│   └── migration_del.error

├── hash

│   ├── migration_hash.log

│   └── migration_hash.success

│   └── migration_hash.exists

│   └── migration_hash.error

├── list

│   ├── migration_list.error

│   ├── migration_list.log

│   └── migration_list.success

├── Set

│  ├── migration_Set.exists

│  ├── migration_Set.log

│  └── migration_Set.success

│  └── migration_Set.error

└── sSet

│  ├── migration_sSet.log

│  └── migration_sSet.success

│  └── migration_sSet.error

├── expire

│   ├── migration_expire.error

│   ├── migration_expire.log

│   └── migration_expire.success

└── string

    ├── migration_string.log

    └── migration_string.success

    └── migration_string.error

1.  bin/redis_data_migration.sh：执行脚本（支持string、hash、list、Set集合、有序集合(sorted set)迁移数据到Redis Cluster，还支持批量删除key值，添加过期时间等）

2.  conf/redis_data_migration.conf：全局参数配置文件

2.1原key值前缀、key值操作类型[string/hash/list/Set/sSet/del]、公有路径、原Redis服务器IP&PORT文件、Redis Cluster IP&PORT文件、Redis Cluster 连接节点IP、Redis Cluster 连接PORT、老 Redis 连接密码[jy34@MLshuoMG]，没有设为空等参数，KEY值过期设置等

2.2 过期设置类型：

     1 EXPIRE key seconds　　//将key的生存时间设置为ttl秒

     2 PEXPIRE key milliseconds　　//将key的生成时间设置为ttl毫秒

     3 EXPIREAT key timestamp　　//将key的过期时间设置为timestamp所代表的的秒数的时间戳

     4 PEXPIREAT key milliseconds-timestamp　　//将key的过期时间设置为timestamp所代表的的毫秒数的时间戳

3.  ip_port/migration_redis_ip_port.[project]：原项目对应Redis服务器IP&PORT文件（如higo_bim：migration_redis_ip_port.bim）

4.  ip_port/redis_cluster_ip_port.[project]：Redis Cluster IP&PORT文件（如higo_bim：redis_cluster_ip_port.bim；inf环境：redis_cluster_ip_port.inf）

5.  string、hash、list、Set、sSet、expire目录为在redis_data_migration.conf配置key值操作类型后生成的对应目录，并生成对应的日志文件：

5.1 migration_[key_type].log：日志文件；   

类型

举个栗子

string

IP:PORT=>xx.xx.1.24:6379##key=>wealthservice:bim_image_id_xxxxx##value=>a:10:{s:2:"id";i:113113;s:8:"image_id";i:xxxxx;s:10:"image_path";s:62:"http://xxxx.xxxx.com/xxx/54654b016368aa6f0d13b2efd3d45160.jpg";s:10:"image_hash";s:28:"Ft7VHRmd5S5ZvKdTqkvVz0LVqnmo";s:11:"image_width";i:900;s:12:"image_height";i:1200;s:10:"image_size";i:289858;s:12:"image_status";i:1;s:11:"image_mtime";s:19:"2018-12-02 17:54:31";s:11:"image_ctime";s:19:"2018-12-02 17:54:31";}##status=>success.

hash

IP:PORT=>xx.xx.1.24:6379##key=>wealthservice:xxx_new_xxxxx##field=>2018112121_2805xxxxx_new##value=>0.04##hexists=>0##status=>success.

list

IP:PORT=>xx.xx.0.47:6489##key=>xxxxim:ImMsgQueueShop:2:sync472xxx525##value=>{"msg_id":"1567414277","uid":3,"shop_id":0}##status=>lrem error.

IP:PORT=>xx.xx.0.40:6479##key=>xxxxim:ImMsgQueueShop:2:sync425xxx979##value=>{"msg_id":1746206575,"uid":"1200882445","shop_id":0}##status=>success.

IP:PORT=>xx.xx.0.47:6489##key=>xxxxim:ImMsgQueueShop:2:sync426xxx901##value=>{"msg_id":"1588692513","uid":1200762040,"shop_id":0}##status=>lrem error.

Set

IP:PORT=>xx.xx.1.22:6379##key=>wealthservice:t_bim_fw_followers_openIds##member=>orwuH1HRTMhk_td3EpWrSW4fX-Io##sismember=>0##status=>success.
sSet IP:PORT=>xx.xx.1.24:6379##key=>bimuserservice:t_bim_most_publish_2018-12-11##score=>3##member=>2318608xxxxx653901##status=>success.

5.2 migration_[key_type].success：迁移成功的执行命令日志；

类型

举个栗子

string

redis-cli -h xx.xx.2.35 -c -p 7001 setnx wealthservice:bim_image_id_xxxxx 'a:10:{s:2:"id";i:113113;s:8:"image_id";i:2814692718xxxxx980;s:10:"image_path";s:62:"http://xxxx.xxxx.com/xxx/54654b016368aa6f0d13b2efd3d45160.jpg";s:10:"image_hash";s:28:"Ft7VHRmd5S5ZvKdTqkvVz0LVqnmo";s:11:"image_width";i:900;s:12:"image_height";i:1200;s:10:"image_size";i:289858;s:12:"image_status";i:1;s:11:"image_mtime";s:19:"2018-12-02 17:54:31";s:11:"image_ctime";s:19:"2018-12-02 17:54:31";}'

hash

redis-cli -h xx.xx.2.35 -c -p 7001 hset wealthservice:bim_new_2463250059xxxxx923 2018112121_280532297xxxxx6946711_new '0.04'

list

redis-cli -h xx.xx.2.35 -c -p 7001 lrem higoim:ImMsgQueueShop:2:sync42662xx01 0 '{"msg_id":"1624xxx093","uid":1200xxx315,"shop_id":0}'
redis-cli -h xx.xx.2.35 -c -p 7001 rpush higoim:ImMsgQueueShop:2:sync42662xx01 '{"msg_id":"1624xxx093","uid":1200xxx315,"shop_id":0}'

Set

redis-cli -h xx.xx.2.35 -c -p 7001 sadd wealthservice:t_bim_fw_followers_openIds 'orwuH1HRTMhk_td3EpWrSW4fX-Io'
sSet redis-cli -h xx.xx.2.35 -c -p 7001 zadd bimuserservice:t_bim_most_publish_2018-12-11 3 '231860805xxxxx3901'

5.3 migration_[key_type].error：迁移失败的执行命令日志（同5.2）；

5.4 migration_[key_type].exists：迁移中重复值的执行命令日志，如hash中字段是否存在，Set中member元素是否是集合 key 的成员（同5.2）；

二、数据丢失问题

migration_[key_type].[status]日志文件很好的解决了执行失败等数据丢失问题，在提前双写或者老的业务停止的前提下，迁移过程绝对不会丢失任何数据，只要老的Redis中有的，Redis Cluster中就一定会有。

三、执行脚本redis_data_migration.sh

#!/bin/bash
# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
# Program : Redis 数据迁移脚本                                    #
# Redis Cluster : redis-4.0.10                                  #
# Author  : [email protected]                                     #
# Date : 2018-12-11                                             #
# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #

# 更新时间：2018-12-24；更新人：WUFEI；更新内容：增加Redis过期限制方法

# 读取配置文件中的所有变量，设置为全局变量
conf_file="../conf/redis_data_migration.conf"
# 获取Redis Cluster中key值前缀
cluster_prefix=`sed '/^cluster_prefix=/!d;s/.*=//' ${conf_file}`
# 获取原key值前缀
key_prefix=`sed '/^key_prefix=/!d;s/.*=//' ${conf_file}`
# 获取key值类型
key_type=`sed '/^key_type=/!d;s/.*=//' ${conf_file}`
# 获取公有路径
public_path=`sed '/^public_path=/!d;s/.*=//' ${conf_file}`
# 获取原Redis服务器IP&PORT文件
ip_port_file=`sed '/^ip_port_file=/!d;s/.*=//' ${conf_file}`
# 获取所有Redis Cluster节点IP&PORT文件
cluster_ip_file=`sed '/^cluster_ip_file=/!d;s/.*=//' ${conf_file}`
# 获取Redis Cluster IP
cluster_ip=`sed '/^cluster_ip=/!d;s/.*=//' ${conf_file}`
# 获取Redis Cluster PORT
cluster_port=`sed '/^cluster_port=/!d;s/.*=//' ${conf_file}`
# Redis Cluster 连接
cluster_cli="redis-cli -h ${cluster_ip} -c -p ${cluster_port}"
# 获取老 Redis 连接密码
redis_auth=`sed '/^redis_auth=/!d;s/.*=//' ${conf_file}`
# 获取过期过期设置类型
expire_ypte=`sed '/^expire_ypte=/!d;s/.*=//' ${conf_file}`
# 获取过期类型对应形式的时间
expire_ttl=`sed '/^expire_ttl=/!d;s/.*=//' ${conf_file}`

# 创建相关目录
if [ ! -d "${public_path}/${key_type}" ]
then
	mkdir -p ${public_path}/${key_type}
fi

# 判断是否存在密码
if [ -z ${redis_auth} ]
then
	# 如果密码为空设置判断值为0
	redis_auth_judge=0
else
	# 如果密码不为空设置判断值为1
	redis_auth_judge=1
fi

# 迁移string类型的数据
function migration_string(){
	while read ip_port
	do
		old_ip=`echo $ip_port | awk '{print $1}'`
		old_port=`echo $ip_port | awk '{print $2}'`
		# 判断老的Redis连接是否需要添加'-a'参数
		if [ 1 -eq ${redis_auth_judge} ]; then
			old_redis_cli="redis-cli -h ${old_ip} -p ${old_port} -a ${redis_auth}"
		else
			old_redis_cli="redis-cli -h ${old_ip} -p ${old_port}"
		fi
		# 循环获取KEY名
		${old_redis_cli} keys "${key_prefix}*" | while read old_key
		do
			# 判断是否为空
                        if [[ -z ${old_key} ]]
                        then
                                continue
                        fi
			# 获取KEY名对应VALUE值
			old_value=`${old_redis_cli} get ${old_key}`
			# 只有在 key 不存在时设置 key 的值。
			${cluster_cli} setnx ${cluster_prefix}${old_key} "${old_value}"
			import_0k=$?
			if [ 0 -eq "${import_0k}" ]; then
				echo "IP:PORT=>${old_ip}:${old_port}##key=>${cluster_prefix}${old_key}##value=>${old_value}##status=>success." >> ${public_path}/${key_type}/migration_${key_type}.log
				echo "${cluster_cli} setnx ${cluster_prefix}${old_key} '${old_value}'" >> ${public_path}/${key_type}/migration_${key_type}.success
			else
				echo "IP:PORT=>${old_ip}:${old_port}##key=>${cluster_prefix}${old_key}##value=>${old_value}##status=>error." >> ${public_path}/${key_type}/migration_${key_type}.log
				echo "${cluster_cli} setnx ${cluster_prefix}${old_key} '${old_value}'" >> ${public_path}/${key_type}/migration_${key_type}.error
			fi
		done
	done < ${public_path}/${ip_port_file}
}

# 迁移hash类型的数据
function migration_hash(){
	while read ip_port
	do
		old_ip=`echo $ip_port | awk '{print $1}'`
		old_port=`echo $ip_port | awk '{print $2}'`
		# 判断老的Redis连接是否需要添加'-a'参数
		if [ 1 -eq ${redis_auth_judge} ]; then
			old_redis_cli="redis-cli -h ${old_ip} -p ${old_port} -a ${redis_auth}"
		else
			old_redis_cli="redis-cli -h ${old_ip} -p ${old_port}"
		fi
		# 循环获取KEY名
		${old_redis_cli} keys "${key_prefix}*" | while read old_key
		do
			# 判断是否为空
                        if [[ -z ${old_key} ]]
                        then
                                continue
                        fi
			# 循环获取所有哈希表中的字段
			${old_redis_cli} hkeys ${old_key} | while read old_h_keys
			do
				# 获取给定字段的值
				old_h_value=`${old_redis_cli} hget ${old_key} ${old_h_keys}`
				# 判断Redis Cluster中是否存在
				hexists_num=`${cluster_cli} hexists ${cluster_prefix}${old_key} "${old_h_keys}"`
				if [ 0 -eq "${hexists_num}" ];then
					# 如果不存在则插入
					${cluster_cli} hset ${cluster_prefix}${old_key} "${old_h_keys}" "${old_h_value}"
					import_0k=$?
					if [ 0 -eq "${import_0k}" ]; then
						echo "IP:PORT=>${old_ip}:${old_port}##key=>${cluster_prefix}${old_key}##field=>${old_h_keys}##value=>${old_h_value}##hexists=>${hexists_num}##status=>success." >> ${public_path}/${key_type}/migration_${key_type}.log
						echo "${cluster_cli} hset ${cluster_prefix}${old_key} ${old_h_keys} '${old_h_value}'" >> ${public_path}/${key_type}/migration_${key_type}.success
					else
						echo "IP:PORT=>${old_ip}:${old_port}##key=>${cluster_prefix}${old_key}##field=>${old_h_keys}##value=>${old_h_value}##hexists=>${hexists_num}##status=>error." >> ${public_path}/${key_type}/migration_${key_type}.log
						echo "${cluster_cli} hset ${cluster_prefix}${old_key} ${old_h_keys} '${old_h_value}'" >> ${public_path}/${key_type}/migration_${key_type}.error
					fi
				else
					echo "${cluster_cli} hset ${cluster_prefix}${old_key} ${old_h_keys} '${old_h_value}'" >> ${public_path}/${key_type}/migration_${key_type}.exists
					echo "IP:PORT=>${old_ip}:${old_port}##key=>${cluster_prefix}${old_key}##field=>${old_h_keys}##value=>${old_h_value}##hexists=>${hexists_num}##status=>is exists." >> ${public_path}/${key_type}/migration_${key_type}.log
				fi
			done
		done
	done < ${public_path}/${ip_port_file}
}

# 迁移list类型的数据
function migration_list(){
	while read ip_port
	do
		old_ip=`echo $ip_port | awk '{print $1}'`
		old_port=`echo $ip_port | awk '{print $2}'`
		# 判断老的Redis连接是否需要添加'-a'参数
		if [ 1 -eq ${redis_auth_judge} ]; then
			old_redis_cli="redis-cli -h ${old_ip} -p ${old_port} -a ${redis_auth}"
		else
			old_redis_cli="redis-cli -h ${old_ip} -p ${old_port}"
		fi
		# 循环获取KEY名
		${old_redis_cli} keys "${key_prefix}*" | while read old_key
		do
			# 判断是否为空
                        if [[ -z ${old_key} ]]
                        then
                                continue
                        fi
			# 获取列表所以的元素
			${old_redis_cli} lrange ${old_key} 0 -1 | while read old_value
			do
				# 判断元素是否为空
				if [[ -z ${old_value} ]]
				then
					continue
				fi
				# 移除列表与${old_value}想等的所有元素
				${cluster_cli} lrem ${cluster_prefix}${old_key} 0 "${old_value}"
				import_0k=$?
				if [ 0 -eq "${import_0k}" ]; then
					${cluster_cli} rpush ${cluster_prefix}${old_key} "${old_value}"
					import_1k=$?
					if [ 0 -eq "${import_1k}" ]; then
						echo "IP:PORT=>${old_ip}:${old_port}##key=>${cluster_prefix}${old_key}##value=>${old_value}##status=>success." >> ${public_path}/${key_type}/migration_${key_type}.log
						echo "${cluster_cli} lrem ${cluster_prefix}${old_key} 0 '${old_value}'" >> ${public_path}/${key_type}/migration_${key_type}.success
						echo "${cluster_cli} rpush ${cluster_prefix}${old_key} '${old_value}'" >> ${public_path}/${key_type}/migration_${key_type}.success
					else
						echo "IP:PORT=>${old_ip}:${old_port}##key=>${cluster_prefix}${old_key}##value=>${old_value}##status=>rpush error." >> ${public_path}/${key_type}/migration_${key_type}.log
						echo "${cluster_cli} lrem ${cluster_prefix}${old_key} 0 '${old_value}'" >> ${public_path}/${key_type}/migration_${key_type}.error
						echo "${cluster_cli} rpush ${cluster_prefix}${old_key} '${old_value}'" >> ${public_path}/${key_type}/migration_${key_type}.error
					fi
				else
					echo "IP:PORT=>${old_ip}:${old_port}##key=>${cluster_prefix}${old_key}##value=>${old_value}##status=>lrem error." >> ${public_path}/${key_type}/migration_${key_type}.log
					echo "${cluster_cli} lrem ${cluster_prefix}${old_key} 0 '${old_value}'" >> ${public_path}/${key_type}/migration_${key_type}.error
					echo "${cluster_cli} rpush ${cluster_prefix}${old_key} '${old_value}'" >> ${public_path}/${key_type}/migration_${key_type}.error
				fi
			done
		done
	done < ${public_path}/${ip_port_file}
}

# 迁移Set集合类型的数据
function migration_Set(){
	while read ip_port
	do
		old_ip=`echo $ip_port | awk '{print $1}'`
		old_port=`echo $ip_port | awk '{print $2}'`
		# 判断老的Redis连接是否需要添加'-a'参数
		if [ 1 -eq ${redis_auth_judge} ]; then
			old_redis_cli="redis-cli -h ${old_ip} -p ${old_port} -a ${redis_auth}"
		else
			old_redis_cli="redis-cli -h ${old_ip} -p ${old_port}"
		fi
		# 循环获取KEY名
		${old_redis_cli} keys "${key_prefix}*" | while read old_key
		do
			# 判断是否为空
                        if [[ -z ${old_key} ]]
                        then
                                continue
                        fi
			# 返回集合中的所有成员
			${old_redis_cli} smembers ${old_key} | while read old_member
			do
				# 判断member是否为空
				if [[ -z ${old_member} ]]
				then
					continue
				fi
				# 判断 member 元素是否是Redis Cluster集合 key 的成员
				sismember_judge=`${cluster_cli} sismember ${cluster_prefix}${old_key} "${old_member}"`
				if [[ 0 -eq "${sismember_judge}" ]];then
					# 如果不存在则插入
					${cluster_cli} sadd ${cluster_prefix}${old_key} "${old_member}"
					import_0k=$?
					if [ 0 -eq "${import_0k}" ]; then
						echo "IP:PORT=>${old_ip}:${old_port}##key=>${cluster_prefix}${old_key}##member=>${old_member}##sismember=>${sismember_judge}##status=>success." >> ${public_path}/${key_type}/migration_${key_type}.log
						echo "${cluster_cli} sadd ${cluster_prefix}${old_key} '${old_member}'" >> ${public_path}/${key_type}/migration_${key_type}.success
					else
						echo "IP:PORT=>${old_ip}:${old_port}##key=>${cluster_prefix}${old_key}##member=>${old_member}##sismember=>${sismember_judge}##status=>error." >> ${public_path}/${key_type}/migration_${key_type}.log
						echo "${cluster_cli} sadd ${cluster_prefix}${old_key} '${old_member}'" >> ${public_path}/${key_type}/migration_${key_type}.error
					fi
				else
					echo "IP:PORT=>${old_ip}:${old_port}##key=>${cluster_prefix}${old_key}##member=>${old_member}##sismember=>${sismember_judge}##status=>exists." >> ${public_path}/${key_type}/migration_${key_type}.log
					echo "${cluster_cli} sadd ${cluster_prefix}${old_key} '${old_member}'" >> ${public_path}/${key_type}/migration_${key_type}.exists
				fi
			done
		done
	done < ${public_path}/${ip_port_file}
}

# 迁移有序集合类型的数据
function migration_sSet(){
	while read ip_port
	do
		old_ip=`echo $ip_port | awk '{print $1}'`
		old_port=`echo $ip_port | awk '{print $2}'`
		# 判断老的Redis连接是否需要添加'-a'参数
		if [ 1 -eq ${redis_auth_judge} ]; then
			old_redis_cli="redis-cli -h ${old_ip} -p ${old_port} -a ${redis_auth}"
		else
			old_redis_cli="redis-cli -h ${old_ip} -p ${old_port}"
		fi
		# 循环获取KEY名
		${old_redis_cli} keys "${key_prefix}*" | while read old_key
		do
			# 判断是否为空
			if [[ -z ${old_key} ]]
			then
				continue
			fi
			# 返回集合中的所有成员
			${old_redis_cli} zrange ${old_key} 0 -1 | while read old_member
			do
				# 判断member是否为空
				if [[ -z ${old_member} ]]
				then
					continue
				fi
				# 返回有序集中，成员的分数值
				zscore_key_member=`${old_redis_cli} zscore ${old_key} "${old_member}"`
				if [[ 0 -le "${zscore_key_member}" ]];then
					${cluster_cli} zadd ${cluster_prefix}${old_key} ${zscore_key_member} "${old_member}"
					import_0k=$?
					if [ 0 -eq "${import_0k}" ]; then
						echo "IP:PORT=>${old_ip}:${old_port}##key=>${cluster_prefix}${old_key}##score=>${zscore_key_member}##member=>${old_member}##status=>success." >> ${public_path}/${key_type}/migration_${key_type}.log
						echo "${cluster_cli} zadd ${cluster_prefix}${old_key} ${zscore_key_member} '${old_member}'" >> ${public_path}/${key_type}/migration_${key_type}.success
					else
						echo "IP:PORT=>${old_ip}:${old_port}##key=>${cluster_prefix}${old_key}##score=>${zscore_key_member}##member=>${old_member}##status=>error." >> ${public_path}/${key_type}/migration_${key_type}.log
						echo "${cluster_cli} zadd ${cluster_prefix}${old_key} ${zscore_key_member} '${old_member}'" >> ${public_path}/${key_type}/migration_${key_type}.error
					fi
				else
					echo "IP:PORT=>${old_ip}:${old_port}##key=>${cluster_prefix}${old_key}##score=>${zscore_key_member}##member=>${old_member}##status=>error." >> ${public_path}/${key_type}/migration_${key_type}.log
					echo "${cluster_cli} zadd ${cluster_prefix}${old_key} ${zscore_key_member} '${old_member}'" >> ${public_path}/${key_type}/migration_${key_type}.error
				fi
			done
		done
	done < ${public_path}/${ip_port_file}
}

# 批量删除数据
function migration_del(){
	while read ip_port
	do
		cluster_ip=`echo $ip_port | awk '{print $1}'`
		cluster_port=`echo $ip_port | awk '{print $2}'`
		redis_cluster_cli="redis-cli -h ${cluster_ip} -c -p ${cluster_port}"
		# 循环获取KEY名
		${redis_cluster_cli} keys "${cluster_prefix}${key_prefix}*" | while read cluster_key
		do
			# 判断是否为空
			if [[ -z ${cluster_key} ]]
			then
				continue
			fi
			# 删除key值
			${redis_cluster_cli} del "${cluster_key}"
			import_0k=$?
			if [ 0 -eq "${import_0k}" ]; then
				echo "IP:PORT=>${cluster_ip}:${cluster_port}##key=>${cluster_key}##status=>del success." >> ${public_path}/${key_type}/migration_${key_type}.log
				echo "${redis_cluster_cli} del ${cluster_key}" >> ${public_path}/${key_type}/migration_${key_type}.success
			else
				echo "IP:PORT=>${cluster_ip}:${cluster_port}##key=>${cluster_key}##status=>del error." >> ${public_path}/${key_type}/migration_${key_type}.log
				echo "${redis_cluster_cli} del ${cluster_key}" >> ${public_path}/${key_type}/migration_${key_type}.error
			fi
		done
	done < ${public_path}/${cluster_ip_file}
}

# KEY值过期限制
function migration_expire(){
	# 1 EXPIRE key seconds　　//将key的生存时间设置为ttl秒
	# 2 PEXPIRE key milliseconds　　//将key的生成时间设置为ttl毫秒
	# 3 EXPIREAT key timestamp　　//将key的过期时间设置为timestamp所代表的的秒数的时间戳
	# 4 PEXPIREAT key milliseconds-timestamp　　//将key的过期时间设置为timestamp所代表的的毫秒数的时间戳
	# 1、2两种方式是设置一个过期的时间段，就是咱们处理验证码最常用的策略，设置三分钟或五分钟后失效，把分钟数转换成秒或毫秒存储到Redis中。
	# 3、4两种方式是指定一个过期的时间 ，比如优惠券的过期时间是某年某月某日，只是单位不一样。
	# # 返回值
	# # 一个整数值1或0，如果成功地为该键设置了超时时间，返回 1,如果键不存在或无法设置超时时间，返回 0.
	while read ip_port
	do
		cluster_ip=`echo $ip_port | awk '{print $1}'`
		cluster_port=`echo $ip_port | awk '{print $2}'`
		redis_cluster_cli="redis-cli -h ${cluster_ip} -c -p ${cluster_port}"
		# 循环获取KEY名
		${redis_cluster_cli} keys "${cluster_prefix}${key_prefix}*" | while read cluster_key
		do
			# 判断是否为空
			if [[ -z ${cluster_key} ]]
			then
				continue
			fi
			# 判断是否已经设置了过期时间
			judge_ttl=`${redis_cluster_cli} ttl ${cluster_key}`
			if [[ ${judge_ttl} -gt 0 ]]
			then
				continue
			fi
			# 判断过期设置类型,并设置过期时间
			${redis_cluster_cli} ${expire_ypte} ${cluster_key} ${expire_ttl}
			import_0k=$?
			if [ 0 -eq "${import_0k}" ]; then
				echo "IP:PORT=>${cluster_ip}:${cluster_port}##key=>${cluster_key}##status=>${expire_ypte} success." >> ${public_path}/${key_type}/migration_${key_type}.log
				echo "${redis_cluster_cli} ${expire_ypte} ${cluster_key} ${expire_ttl}" >> ${public_path}/${key_type}/migration_${key_type}.success
			else
				echo "IP:PORT=>${cluster_ip}:${cluster_port}##key=>${cluster_key}##status=>${expire_ypte} error." >> ${public_path}/${key_type}/migration_${key_type}.log
				echo "${redis_cluster_cli} ${expire_ypte} ${cluster_key} ${expire_ttl}" >> ${public_path}/${key_type}/migration_${key_type}.error
			fi
		done
	done < ${public_path}/${cluster_ip_file}
}

function main(){
	if [ ${key_type} = 'string' ]
	then
		# 迁移string类型的数据
		migration_string
	elif [ ${key_type} = 'hash' ]
	then
		# 迁移hash类型的数据
		migration_hash
	elif [ ${key_type} = 'list' ]
	then
		# 迁移list类型的数据
		migration_list
	elif [ ${key_type} = 'Set' ]
	then
		# 迁移Set集合类型的数据
		migration_Set
	elif [ ${key_type} = 'sSet' ]
	then
		# 迁移有序集合类型的数据
		migration_sSet
	elif [ ${key_type} = 'del' ]
	then
		# 批量删除数据
		migration_del
	elif [ ${key_type} = 'expire' ]
	then
		# KEY值过期限制
		migration_expire
	else
		# 配置文件错误
		echo "配置文件${conf_file}参数错误，请仔细核对相应参数!"
	fi
}

main

四、全局参数配置文件redis_data_migration.conf

# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
# Program : Redis 数据迁移配置文件                                 #
# Author  : [email protected]                                     #
# Date : 2018-12-11                                             #
# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
# Redis Cluster中key值前缀
cluster_prefix=PUSH:1.106102
# 原key值前缀
key_prefix=
# key值操作类型
key_type=del
# 公有路径
public_path=/home/work/wufei/migration
# 原Redis服务器IP&PORT文件
ip_port_file=ip_port/migration_redis_ip_port.push
# Redis Cluster IP&PORT文件
cluster_ip_file=ip_port/redis_cluster_ip_port.push
# Redis Cluster 连接节点IP
cluster_ip=xx.xx.2.35
# Redis Cluster 连接PORT
cluster_port=7001
# 老 Redis 连接密码(jy34@MLshuoMG)，没有设为空
redis_auth=xxxx@MLxxxxMG
# 过期设置类型
# 1 EXPIRE key seconds　　//将key的生存时间设置为ttl秒
# 2 PEXPIRE key milliseconds　　//将key的生成时间设置为ttl毫秒
# 3 EXPIREAT key timestamp　　//将key的过期时间设置为timestamp所代表的的秒数的时间戳
# 4 PEXPIREAT key milliseconds-timestamp　　//将key的过期时间设置为timestamp所代表的的毫秒数的时间戳
# expire_ypte可设置为expire | pexpire | expireat | pexpireat
# expire_ttl设置为expire_ypte所选择形式的对应时间类型
expire_ypte=expire
expire_ttl=2592000

五、原项目对应Redis服务器IP&PORT文件

（如higo_bim：migration_redis_ip_port.bim）

xx.xx.1.15 6379
xx.xx.1.16 6379
xx.xx.1.18 6379
xx.xx.1.19 6379
xx.xx.1.20 6379
xx.xx.1.21 6379
xx.xx.1.22 6379
xx.xx.1.24 6379

六、Redis Cluster IP&PORT文件

（如higo_bim：redis_cluster_ip_port.bim）

xx.xx.0.12 7121
xx.xx.0.66 7121
xx.xx.0.13 7121
xx.xx.0.71 7121
xx.xx.0.67 7121

猜你喜欢

转载自blog.csdn.net/weixin_42018518/article/details/88815157
今日推荐
开源日报 | Chrome内置Gemini的意义不在于Gemini；中国AI追随之路的五大误区；ECharts创始人“下海”养鱼；谷歌I/O开发者大会什么都有，只是没有惊喜
微软回应中国区AI团队“打包赴美”传闻
基于大语言模型的开源知识库问答系统 MaxKB GitHub Star 数量突破 5,000 个！
美国拟限制 AI 大模型出口中国和俄罗斯
苹果将与 OpenAI 达成协议，将 ChatGPT 应用于 iPhone
openKylin 社区生态委员会第六次会议圆满召开
阿里云正式发布通义千问 2.5
Python 3.13 发布首个 Beta：实验性自由线程模式和 JIT、改进交互式解释器
Stack Overflow 拿我的代码去训练 AI 大模型，还封了我的账号
Pop!_OS 的 COSMIC 桌面完成 App Store 上架工作
《2024 年一季度互联网投融资运行情况》研究报告
报告：Django 仍然是 74% 开发者的首选
周排行
返回指定时间格式
fopen函数中的mode参数
Java 单例模式探讨
Flex remoteobject工作原理探讨
寻找mplayer的便捷安装方法
30天了解30种技术系列---(26)MySQL自动化运维工具Inception
关于Jboss/Tomcat/Jetty的JNDI定义123
程序减肥，strip，eu-strip 及其符号表
AsyncTask、View.post(Runnable)、ViewTreeObserver三种方式总结frame animation自动启动
Json和Bean的互相转换
每日归档
更多
2024-05-15(24)
2024-05-14(0)
2024-05-13(18)
2024-05-12(0)
2024-05-11(38)
2024-05-10(38)
2024-05-09(35)
2024-05-08(42)
2024-05-07(14)
2024-05-06(40)

代码天地 © 2018 codetd.com

境外服务器   最新电视剧2022