Redis single node capacity problem
A single node capacity issues
We in the actual scene, often encounter a single node capacity issues.
1. service splitter , data classification
2. to the data can not be split when sliced data can be
- Modulo hash (under the influence of a distributed scalable 3%, 4%, if a machine pay more, will receive the impact)
- Random logic (can be put to, but get it out)
- Solution: two machines simultaneously store a list, then the client directly to connect two redis, carried out with two consumer
- Consistent hashing algorithm
- crc16 crc32 md5 sha1 sha256
- No modulo-width 16-bit, 16-bit hash abstract a ring, to calculating hashing algorithm
一致性哈希算法(哈希环):
1.求出memcached服务器(节点)的哈希值,并将其配置到0~232的圆(continuum)上。
2.采用同样的方法求出存储数据的键的哈希值,并映射到相同的圆上。
3.从数据映射到的位置开始顺时针查找,将数据保存到找到的第一个服务器上。如果超过232仍然找不到服务器,就会保存到第一台memcached服务器上。
来源:(https://www.cnblogs.com/williamjie/p/9477852.html)
3. advantages and disadvantages
Pros: add node to other nodes can indeed share the pressure (and will not cause a global reshuffle)
Cons: New node will cause a small portion of the data can not hit
二、twemproxy
twemproxy fragmentation agent is a mechanism by open twitter, twemproxy as a proxy can accept multiple access programs, in accordance with the routing rules, the forwarding Redis for the background of each server, then backtrack, the program a good solution Redis examples carrying capacity.
installation
git clone https://github.com/twitter/twemproxy.git
如果报错,执行:yum update nss
yum install automake libtool
autoreconf -fvi
如果报错,执行
wget -O /etc/yum.repos.d/epel.repo http://mirrors.aliyun.com/repo/epel-6.repo
yum clean all
yum install autoconf268.noarch -y
autoreconf268 -fvi
./configure --enable-debug=full
make
查看服务文件
cd scripts
nutcracker.init
拷贝这个文件进/etc/init.d目录
拷贝编译运行文件进/usr/bin目录
拷贝conf文件夹进/etc/nutcracker目录
进入/etc/nutcracker,修改nutcracker.yml进行配置
alpha:
listen: 127.0.0.1:22121
hash: fnv1a_64
distribution: ketama
auto_eject_hosts: true
redis: true
server_retry_timeout: 2000
server_failure_limit: 1
servers:
- 127.0.0.1:6379:1
- 127.0.0.1:6380:1
之后开启nutcracker服务,开启service服务,之后连接redis-cli进行连接22121端口
我们通过nutcracker进行get和set,我们在nutcracker不支持的命令:
keys *
watch k1
multi
这些命令都不支持
predixy software, it can also be used as a substitute
wget https://github.com/joyieldInc/predixy/releases/download/1.0.5/predixy-1.0.5-bin-amd64-linux.tar.gz
Modify predixy.conf
打开Bind 127.0.0.1:7617
打开include sentinel.conf
Modify the Sentinel 26379
port 26379
sentinel monitor ooxx 127.0.0.1 36379 2
sentinel monitor xxoo 127.0.0.1 46379 2
Modify the Sentinel 26380
port 26380
sentinel monitor ooxx 127.0.0.1 36379 2
sentinel monitor xxoo 127.0.0.1 46379 2
The following were started omitted.
After the direct test