Chapter One Introduction to Redis cluster
1.1 Functional Overview
Remote Dictionary Server (Redis) is a key-based key-value database for persistent storage system. Supports a variety of data structures, including string (string), List (list), SET (set), zset (sorted set - ordered set) and hash (hash type). These data types are supported push / pop, add / remove and on the intersection and union, and difference richer operation, and these operations are atomic.
1.2 Functional Description
Redis Redis Cluster is a distributed implementation, mainly in order to achieve the following objectives (in order of importance in the design):
In 1000, when the node can still perform well and scalability is linear.
No merge operation, so that Redis data model is the most typical large data values can have a good performance.
Writing security: a write operation, the system attempts to those customers with most nodes connected to the end made all preserved. But acknowledged there was still a small part is written will be lost.Because ofFrom the master node to complete the synchronization data is transmitted over the network,The master node suddenly crash, will cause no slave nodesProblems synchronizing data.
Availability: In the vast majority of the primary node is reachable, and have at least one of its nodes reachable from the case, Redis cluster partition can still operate for each primary node unreachable.
1.3 deployment instructions
ThatStoring cluster data using fingering slots, three servers six nodes, three master node and three slave nodes from the master node of the master copy implemented completely copied from the associated node. Three hash slot allocated by the master node storage command, fully to the center. Support node cluster expansion and removal operations.
The following is the actual production environment deployment instructions:
| 1 |
Host Name |
The first set of clusters |
The second set of clusters |
Allocate memory |
IP addresses |
| 2 |
jdpsclb-nfppzk01 |
6379 6479 |
6579 6679 |
They agreed to 50G |
172.1.2.3 |
| 3 |
jdpsclb-nfppzk02 |
6379 6479 |
6579 6679 |
They agreed to 50G |
172.1.2.4 |
| 4 |
jdpsclb-nfppzk03 |
6379 6479 |
6579 6679 |
They agreed to 50G |
172.1.2.5 |
In a production environment deployed two redis clusters on three servers. The following documents for the first set of written rdis cluster.
Chapter Redis cluster installation deployment
2.1 Environment Preparation
2.1.1 Install the software ready
repeat-2.3.3
ruby-2.4.0
redis-3.2.1.gem
2.1.2 Environmental inspection
Before installing the rpm command queries run redis dependencies, if not installed dependencies, you need to use yum to install.
Redis query dependencies are installed command:
rpm -q cpp binutils glibc glibc-kernheaders glibc-common glibc-devel gcc make gcc-c++ libstdc++-devel tcl zlib-devel
2.2 Redis stand-alone installation
Note: Use the root user to install when you install, the actual production environment, please create a new user (redis or cache, etc.) management.
2.2.1 Redis installed
- Installation redis dependencies preferably used yum command to install
yum -y install cpp binutils glibc glibc-kernheaders glibc-common glibc-devel gcc make gcc-c++ libstdc++-devel tcl zlib-devel
- Redis installation package to upload a file directory, extract the installation package
Installation path: / home / redis / cluster /
Profile: /home/redis/cluster/6379/redis.conf
takes -xzvf Redis-3.2.9.tar.gz
- redis-3.2.9 directory into the execution-extracting installation command make and make install
make
make install
Into the src directory after the installation is complete, redis-trip.rb copy files to / usr / local / bin (do require root user environment variables must)
cp redis-trib.rb /usr/local/bin
2.2.2 modify the configuration file
Following a user operation using redis
2.2.3 New Profile directory
mkdir -p /home/redis/standalone
2.2.3.1 Copy the configuration file to the new directory and configure.
Copy the configuration file
cp/opt/redis/redis-3.2.9/redis.conf /home/redis/standalone
Modify the configuration file
we /home/redis/standalone/redis.conf
Modifications are as follows:
The port number:
port 6379 (default)
IP binding, modify the server ip:
bind 172.31.87.1
Backup file directory:
dir /home/redis/standalone/
Configuration daemon start
daemonize yes
Modify the process ID file
pidfile /home/redis/standalone/redis.pid
Modify the log file path
logfile "/home/redis/standalone/redis.log"
Modify the number of open database
databases 1
Save and Exit
2.2.3.2 standalone installation check
Use redis user verification
1. Start redis service and view redis process
cd /home/redis/standalone
Specifies the configuration file to start redis redis
redis-server redis.conf
View redis process
ps -ef | grep Redis server
2. Connect client test
Connection redis
-H 172.22.1.155 -p-CLI Redis 6379 (subject to produce a physical address)
Stored value
set key1 value1
The value
get key1
3. Test ok close redis process, redis cluster does not need a separate node
pkill -9 redis-server
2.3 Cluster Installation
2.3.1 prerequisite condition
- Cluster Installation requires redis3.0 + version support
- Complete stand-alone installation on each server
- redis3.x need ruby cluster environment support, we need to install ruby environment
2.3.2 cluster configuration file modifications
Redis user switching
1. Create a directory on the host two nodes need to be deployed redis clusters
mkdir -p /home/redis/cluster/6379
mkdir -p /home/redis/cluster/6479
2. Copy / home / redis / redis.conf in standalone node directory to the directory 6379
3. Edit the file just copied redis.conf
4. Edit the profile contents
Modify the process ID file to the port number to distinguish
Modify pidfile
pidfile /home/redis/cluster/6379/redis.pid
Modify the log file storage directory
logfile “/home/redis/cluster/6379/redis.log”
Enable the cluster
cluster-enabled yes
Specifies the cluster configuration file
cluster-config-file "/home/redis/cluster/6379/nodes.conf"
The nodes in the cluster hung up, does not affect the entire cluster, you can access data from other nodes of normal
cluster-require-full-coverage no
This configuration represents the background start
daemonize yes
The port number:
port 6379 (default)
IP binding, modified to your own server ip:
bind 172.31.87.1
Backup file directory:
dir /home/redis/cluster/6379/
The number of open database
databases 1
5. After the configuration file is modified copy redis.conf just modified file to the directory of each node, and then modify the configuration files of each node in the directory attribute ip bind port and port attribute
2.3.3 install Ruby
- Select a server install ruby to create a cluster redis
- Root privileges to install
- Ruby-2.4.0.tar.gz uploaded to the server, and extract
- Extracting installation package ruby
- tar xf ruby-2.4.0-tar.gz
- Run ./configure perform the installation and configuration detection installation environment
- ./configure
- Compiling and Installing
Compile
make
installation
make install
2.3.4 mounting redis-3.2.1.gem
Redis-3.2.1.gem only installed the machine is equipped with the ruby redis.3.2.1.gem
Upload redis-3.2.1.gem to the next / home / redis installation directory
gem install –-local redis-3.2.1.gem
2.3.5 Each node starts
It was started six nodes on three servers
redis-server /home/redis/cluster/6379/redis.conf
redis-server /home/redis/cluster/6479/redis.conf
After three servers six nodes to check nodes turned on, each server will display two nodes
ps -ef | grep Redis server
2.3.6 create build clusters
Execute commands on the creation of ruby server installation, create the following command:
redis-trib.rb create –-replicas 1 172.21.26.5:6379
172.21.26.5:6479 172.21.26.6:6379 172.21.26.6:6479
172.21.26.7:6379 172.21.26.7:6479
Start creating a cluster, you will enter during the yes or no input yes
See the following figure prompted, create a cluster successful.
2.4 redis clusters optimization
2.4.1 maxmemory Options
Do not let Redis machine where the actual physical memory usage than 3/5 of total memory.
Configuration redis.conf in maxmemory options to configure standard of 40 percent of physical memory.
If the main points from both nodes 20% respectively arranged on a single server. The following options:
(Allocated to the production environment 200G memory, whichever)
maxmemory 5100000000
Setting Memory elimination mechanism
allkeys-lru: delete key priority recently infrequently used to save the new data
maxmemory-policy allkeys-lru
2.4.2 open TCP connections fast recovery
Open TCP connections fast recovery in TIME-WAIT sockets, contribute to the rapid release of TCP connections
Modify /etc/sysctl.conf file
Modify the following parameters, if not please add this parameter
net.ipv4.tcp_tw_recycle=1
2.5 Verifying the Deployment
2.5.1 Data storage validation
Redis connection, set and get used to store and retrieve data, note storage node information
Connection redis:
redis-cli -c -h 172.22.1.156 (address of actual production date)
Stored value
set key value
The value
get key
Three main nodes have corresponding instructions to create a cluster successful