Redis(7)Creating and Using Cluster Mode
1. Documents
Cluster will not support SELECT, it only contains database 0.
All the nodes use TCP bus and binary protocol to communicate. Every node connect to cluster bus. The use gossip to talk.
Some times we lost writing data
1. master node dies right before the slave node get the newly writing data.
2. master node dies, and slave node become master, and receive a writing data. when the old master recover, it does not have the newly writing data.
2. Configuration the Cluster mode
Check out the latest codes from here https://github.com/antirez/redis.git
Switch to the branch unstable
>make distclean
>make
>mkdir /Users/carl/tool/redis-unstable
>cp redis-server /Users/carl/tool/redis-unstable/
>cp redis-cli /Users/carl/tool/redis-unstable/
>cp redis-benchmark /Users/carl/tool/redis-unstable/
>cd ..
>cp redis.conf /Users/carl/tool/redis-unstable/
>sudo ln -s /Users/carl/tool/redis-unstable /opt/redis-unstable
>sudo rm -fr /opt/redis
>sudo ln -s /opt/redis-unstable /opt/redis
Prepare the Configuration files
>mkdir cluster-conf
>cd cluster-conf
>mkdir 7000 7001 7002 7003 7004 7005 7006
Change the redis.conf there and properties as follow for references;
port 7000
appendonly yes
cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 5000
Then start all the nodes one by one
>../../redis-server ./redis.conf
Every time, we will see these message after we start one node.
[23303] 01 May 15:28:47.602 * No cluster configuration found, I'm c5a855fba006f6b3302f7c162ba3b6a71d548b58
After we start 7000 to 7005, 6 nodes, execute this command from src directory to configure cluster.
>./redis-trib.rb create --replicas 1 127.0.0.1:7000 127.0.0.1:7001 127.0.0.1:7002 127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005
Error Message:
cannot load such file -- redis (LoadError)
Solution:
>ruby -v
ruby 2.0.0p451 (2014-02-24 revision 45167) [universal.x86_64-darwin13]
>gem -v
2.0.14
1. Use a recent ruby (1.9.3 or 2.0)
2. Install the redid gem
>sudo gem install redis
Then
>ruby -rubygems ./redis-trib.rb create --replicas 1 127.0.0.1:7000 127.0.0.1:7001 127.0.0.1:7002 127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005
>>> Creating cluster Connecting to node 127.0.0.1:7000: OK Connecting to node 127.0.0.1:7001: OK Connecting to node 127.0.0.1:7002: OK Connecting to node 127.0.0.1:7003: OK Connecting to node 127.0.0.1:7004: OK Connecting to node 127.0.0.1:7005: OK >>> Performing hash slots allocation on 6 nodes... Using 3 masters: 127.0.0.1:7000 127.0.0.1:7001 127.0.0.1:7002 Adding replica 127.0.0.1:7003 to 127.0.0.1:7000 Adding replica 127.0.0.1:7004 to 127.0.0.1:7001 Adding replica 127.0.0.1:7005 to 127.0.0.1:7002 M: c5a855fba006f6b3302f7c162ba3b6a71d548b58 127.0.0.1:7000 slots:0-5460 (5461 slots) master M: d0a6d910adeffa2b0ba69ddc69d9e69bdda1a41e 127.0.0.1:7001 slots:5461-10922 (5462 slots) master M: 376e13f055d4119ca45824aff3e7722a3043ecf7 127.0.0.1:7002 slots:10923-16383 (5461 slots) master S: 5295fe0fae941d7f964a674c5e5a4de37738e412 127.0.0.1:7003 replicates c5a855fba006f6b3302f7c162ba3b6a71d548b58 S: 64a3ae73e370d1e016a01adaedce1606d3b7d2ea 127.0.0.1:7004 replicates d0a6d910adeffa2b0ba69ddc69d9e69bdda1a41e S: 4324a0c7c90a460b4fc107f66e61656ec23c316b 127.0.0.1:7005 replicates 376e13f055d4119ca45824aff3e7722a3043ecf7 Can I set the above configuration? (type 'yes' to accept): yes >>> Nodes configuration updated >>> Sending CLUSTER MEET messages to join the cluster Waiting for the cluster to join... >>> Performing Cluster Check (using node 127.0.0.1:7000) M: c5a855fba006f6b3302f7c162ba3b6a71d548b58 127.0.0.1:7000 slots:0-5460 (5461 slots) master M: d0a6d910adeffa2b0ba69ddc69d9e69bdda1a41e 127.0.0.1:7001 slots:5461-10922 (5462 slots) master M: 376e13f055d4119ca45824aff3e7722a3043ecf7 127.0.0.1:7002 slots:10923-16383 (5461 slots) master M: 5295fe0fae941d7f964a674c5e5a4de37738e412 127.0.0.1:7003 slots: (0 slots) master replicates c5a855fba006f6b3302f7c162ba3b6a71d548b58 M: 64a3ae73e370d1e016a01adaedce1606d3b7d2ea 127.0.0.1:7004 slots: (0 slots) master replicates d0a6d910adeffa2b0ba69ddc69d9e69bdda1a41e M: 4324a0c7c90a460b4fc107f66e61656ec23c316b 127.0.0.1:7005 slots: (0 slots) master replicates 376e13f055d4119ca45824aff3e7722a3043ecf7 [OK] All nodes agree about slots configuration. >>> Check for open slots... >>> Check slots coverage... [OK] All 16384 slots covered.
—replicas 1 indicate that every master has a replica node.
It will create 3 masters and 3 slaves.
Right now, my version of branch unstable is 2.9.11.
Test and Verify the Cluster
>./redis-cli -c -p 7000
127.0.0.1:7000> set foo bar -> Redirected to slot [12182] located at 127.0.0.1:7002 OK 127.0.0.1:7002> get foo "bar" 127.0.0.1:7002> set hello world -> Redirected to slot [866] located at 127.0.0.1:7000 OK 127.0.0.1:7000> get hello "world"
Add New Node
redis>cluster nodes
This command will show all the nodes.
>./redis-trib.rb add-node 127.0.0.1:7006 127.0.0.1:7000
Connecting to node 127.0.0.1:7006: OK >>> Send CLUSTER MEET to node 127.0.0.1:7006 to make it join the cluster. [OK] New node added correctly.
Or add slave node
>./redis-trib.rb add-node --slave 127.0.0.1:7006 127.0.0.1:7000
Or
>./redis-trib.rb add-node --slave --master-id adsfadsfadsfsadfsdaf 127.0.0.1:7006 127.0.0.1:7000
Remove a Node
>./redis-trib.rb del-node 127.0.0.1:7000 '591d10c5251abd0ef655a458f8e16a6cca182da6'
First parameter is any node host and port, the second parameter the id of the node we plan to remove.
Error Message:
[ERR] Node 127.0.0.1:7006 is not empty. Either the node already knows other nodes (check with CLUSTER NODES) or contains some key in database 0.
Solution:
I delete the node, I plan to add it again. But I meet that error message.
I do not know how to fix this right now. But an easy way, delete all the files under 7006 directory except redis.conf, they it will generate a new node id.
3. Scala Client
Check the Status
redis>info
redis_mode:cluster
redis_version:2.9.11
Open project easynosqlscala
It runs great.
package com.sillycat.easynosqlscala.app
import redis.clients.jedis.{ JedisCluster, HostAndPort, Jedis }
import java.util.HashSet
object TestRedisDBConnectionApp extends App {
// val jedis = new Jedis("172.0.0.1", 7000)
// jedis.set("name", "sillycat")
// println(jedis.get("name"))
val jedisClusterNodes = new HashSet[HostAndPort]
jedisClusterNodes.add(new HostAndPort("127.0.0.1", 7000))
jedisClusterNodes.add(new HostAndPort("127.0.0.1", 7001))
jedisClusterNodes.add(new HostAndPort("127.0.0.1", 7002))
jedisClusterNodes.add(new HostAndPort("127.0.0.1", 7003))
jedisClusterNodes.add(new HostAndPort("127.0.0.1", 7004))
jedisClusterNodes.add(new HostAndPort("127.0.0.1", 7005))
jedisClusterNodes.add(new HostAndPort("127.0.0.1", 7006))
val jc = new JedisCluster(jedisClusterNodes)
jc.set("age", "32")
println("name = " + jc.get("name") + " age = " + jc.get("age"))
}
Redis Monitor
https://github.com/LittlePeng/redis-monitor
References:
Redis 1~ 6
http://sillycat.iteye.com/blog/1549504 Installation redis on windows, ubuntu, redhat
http://sillycat.iteye.com/blog/1553507 Data Type, String, Int, List, Set
http://sillycat.iteye.com/blog/1553508 Data Type, Ordered Set, Hash, Publish/Subscribe, Data Expiration
http://sillycat.iteye.com/blog/1553509 Java DAO Layer
http://sillycat.iteye.com/blog/2028180 Installation on MAC, HA Solution sentinel master slave
http://sillycat.iteye.com/blog/2033094 Scala Client
http://redis.io/topics/cluster-tutorial
http://redis.io/topics/cluster-spec
http://www.redis.cn/topics/cluster-tutorial.html
http://www.redis.cn/topics/cluster-spec.html
Tips
http://in.sdo.com/?p=1378
http://www.cnblogs.com/lulu/archive/2013/06/10/3130878.html
http://www.oschina.net/translate/utopian-redis-cluster
cluster
http://hot66hot.iteye.com/blog/2050676