When a cluster, the at least consider the following factors:
(1) high availability requirements: The principle of failover, the primary node requires at least three to complete failover, and 3 are not the master node on the same physical machine; Each the master node at least one slave node, the master and the slave node should not be a single physical machine; therefore high-availability cluster comprising at least six nodes.
(2) the amount of data and the views: estimating the amount of data required by the application totals (business considerations, leaving redundant), binding capacity, and can withstand the amount of each primary access node (which may be obtained by accurately benchmark estimation), calculates the required number of master node.
Number (3) node limit: limit the number of nodes official gives Redis 1000, mainly caused by consuming inter-node communication considerations. In practice, we should try to avoid large clusters; if the number of nodes is not sufficient to meet the application requirements of Redis data volume and traffic, consider:
. A business division, large clusters into smaller clusters;
. B reduce unnecessary transactions;
c. Adjust the data expiration policies.
(4) moderate redundancy: Redis can add nodes in the cluster without affecting services, the number of nodes can be appropriate redundancy, not too much.
Clusters of principle:
The core functions of the data partition clusters, thus introduces data partitioning rule; describes the details of the cluster is then implemented: the communication mechanism and data structures; Finally cluster meet (node handshake), cluster addslots (channel allocation), for example, indicates that the node It is how to use the above-described data structures and communication mechanisms to achieve the cluster command.
Data partitioning scheme:
data partition sequential partition, the hash partitioning, wherein the hash partitioning due to its natural randomness, are widely used; cluster partitioning scheme is a kind of hash partitions.
Hash partitioning basic idea is: The feature value data (e.g., key) hash, then decide which node data according to the hash value falls. Common hash partitioning include: a hash modulo partition, consistency hash partitioning, hash partitioning consistency with virtual node.
(1) modulo hash partitions
hash modulo partition idea is very simple: calculating a hash value of the key, and then modulo the number of nodes, to determine which node on the map data. The program's biggest problem is that, when added to or delete nodes, change the number of nodes, all data in the system will need to re-calculate the mapping relationship, triggering large-scale data migration.
(2) consistent hashing partition
consistent hashing algorithm to hash value of the entire space into a virtual ring, the range of 0-2 ^ 32-1; for each data, a hash value is calculated according to the key, the data determined position on the ring, and then run along the ring clockwise from this position, the first server is found that should be mapped to the server.
Compared with the hash modulo partition, partition will affect the increase or decrease in consistent hashing node is limited to adjacent nodes. If the increase node5 between node1 and node2, node2 only part of the data is migrated to node5; if you remove node2, node2 in the original data will only migrate to node4, only node4 affected.
The main problem is that consistent hashing partition, when the number of nodes, add or delete nodes, the impact on individual nodes can be large, causing serious imbalance data. FIG still above example, if node2 removed, the data node4 by about one quarter of the total data becomes approximately 1/2 load is too high compared with other nodes.
(3) consistent hashing with virtual node partitioning
of the program on the basis of consistent hashing partition on the introduction of the concept of virtual nodes. Redis is used in the cluster program, which is called a virtual node slots (slot). The concept of virtual slot is interposed between the data and the actual node; each actual node contains a certain number of slots, each slot comprising data hash value within a certain range. After the introduction of slots,
Mapping relational data by the data hash-> Actual node data into hash-> groove -> actual node.
Using a consistent hashing partition slot, the slot is the basic unit of data management and migration. Slot decoupling the relationship between the data and the actual node, add or delete nodes influence on the system is minimal. FIG still above as an example, the actual system has four nodes, assuming assigned 16 slots (0-15); 0-3 groove located node1,4-7 located node2,
And so on. If at this time remove node2, grooves only need to redistribute to 4-7, e.g. 4-5 slots allocated to node1, 6 to node3 distribution groove, the groove 7 to Node4; delete node2 can be seen after, other nodes in the data still more balanced distribution. The number of slots is generally much smaller than 2 ^ 32, is much greater than the actual number of nodes;
In Redis cluster, the number of slots 16384
The picture below summarizes the process well Redis cluster mapping data to the actual node:
(1) Redis hash value calculated feature value data (generally key), using the algorithm CRC16. CRC16 (Key) = the hash
(2) The hash value calculation data belong to which channel. 16384% hash
(. 3) according to the mapping of slots to nodes, which nodes belong to the calculated data.
Redis Cluster Setup:
A master / slave (master / slave) (disadvantages: data redundancy, waste of memory (ping - pong)
1. The master reads and writes, can only be read from the node can write
2. When write data changes when the master node, the synchronization directly from node
3 can have a plurality of slave master, but a slave can only correspond to a master
Master-slave replication mechanism:
when the slave starts, the initiative to send SYNC command to the master. After receiving a master SYNC command to save a snapshot (RDB persistence) and caching to save a snapshot of command during this time, and then send the saved snapshot file and cache commands to the slave in the background. After the slave receives the snapshot file and snapshot file and command execution command to load the cache. After copying the initialization, master to each write command received are sent to a synchronization slave, from the master to ensure data consistency.
Master-slave configuration:
Redis default master data, master no configuration, we only need to modify the configuration of slave.
a master is provided to be connected to port ip:.
slaveof 192.168.0.107 6379
. B If the master password has been set. You need to configure:
masterauth
c Once connected into the command line, you can view other library information line connection to the database by the following command:.
Info Replication
Second, Sentinel mode (Sentinel)
Sentinel's command, Sentinel is a separate process, as a process, it will operate independently. The principle is Sentinel by sending a command, the server waits for a response Redis to monitor multiple instances running Redis. (Ping - pong)
Sentinel here has two functions:
• by sending a command, so that Redis server returns monitor its operational status, including primary and secondary servers.
• When the sentinel surveillance to master goes down, it will automatically switch to the slave master, and then notify the other from the server by issuing subscription model, modify the configuration file, so that they switch hosts.
However, a sentry process of Redis server monitoring, problems may occur, for which we can use multiple sentinel monitoring. But also between the various monitoring sentinel, thus forming a multi-Sentinel mode.
Process failover (failover) is:
Assume that the primary server goes down, the first sentinel 1 detects this result, the system does not immediately perform failover process, only a sentinel subjective that primary server is unavailable, offline subjective phenomenon becomes. When the latter also detected in sentinel primary server is unavailable, and the number reaches a certain value, it will be a vote between sentinel, the voting result is initiated by a sentinel, a failover operation. After the successful handover, will be released through a subscription model that allows each sentinel to monitor the realization of their own to switch from the host server, this process is called objective offline. For such clients, everything is transparent
Configuration:
Redis master configuration from the server, modify the file as follows redis.conf
# makes Redis server across the network access
the bind 0.0.0.0
# Set Password
requirepass "123456"
# specify the master server Note: For configuration of slaveof only configuration from the server, the primary server no need to configure
slaveof 192.168.11.128 6379
# master password Note: For configuration of slaveof only configuration from the server, the master server does not require configuration
masterauth 123456
configure Sentinel, there is a sentinel.conf Redis file in the installation directory, copy conduct a modify
# Disable protected mode
protected-the mODE NO
# configure the listening master server, here on behalf of sentinel monitor monitor, on behalf of the server name mymaster, you can customize, 192.168.11.128 representatives to monitor the primary server, 6379 on behalf of the port, 2 for only two two or more sentinel that when the primary server is unavailable, the operation will be failover.
Monitor mymaster 192.168.11.128 6379 2 Sentinel
# author Sentinel Pass-defined service password, mymaster is the service name, Redis server password is 123456
# auth-Sentinel Pass <Master-name> <password>
Sentinel Pass mymaster auth-123456
Start:
# start the Redis server process
./redis-server ../redis.conf
# sentry start the process
./redis-sentinel ../sentinel.conf
Of particular note are:
Objective offline concept unique to the master node; if a fault occurs and sentinel nodes from the node, the sentinel subjectively offline, there will be no subsequent objective offline and failover operation. Sentinel node election leader: When the primary node is judged objectively offline, each sentinel node will consult elected a leader sentinel node by node be the leader failover operation. The master node monitors all guards are likely to be elected leader election algorithm using a Raft algorithm; Raft basic idea of the algorithm is first come first served: that in an election, Sentinel A to B to send to be a leader the application, if B had not agreed to the other sentry, it will be a leader a consent. Specific electoral process not described in detail here, in general, the selection process sentry quickly, who should complete and objective off the assembly line, the general will be able to be a leader.
Failover: elected leader Sentinel, start failover operation, which can be roughly divided into three steps:
• In selecting a new primary node from the node: the principle of choice is to first filter out unhealthy from the node; then select the highest priority (designated by the slave-priority) from the node; if the priority can not be distinguished, then select Copy the maximum offset from the node; if still can not be distinguished from the minimum runid selected node.
• master-slave status update: The slaveof no one command, so that from the node becomes the master node elected; and let the other nodes through slaveof order to become its slave nodes.
• the master node (i.e., 6379) has been offline set as a new master node from the node 6379 when the back online, it will become the new node from the master node.
Third, clusters
Clusters, namely Redis Cluster, distributed storage solution Redis 3.0 is introduced in the beginning.
Cluster consists of a plurality of nodes (Node) composition, Redis data distributed among the nodes. Sub-nodes in the cluster master node and slave node: only the master node is responsible for maintaining read and write requests and cluster information; duplicate only the master node data and status information from the node.
The role of the cluster, can be summarized in two points:
1, data partitioning: data partition (also known as data fragmentation) is the core cluster functionality.
The cluster data across multiple nodes, on the one hand breaks through the limit Redis stand-alone memory size, storage capacity greatly increased; on the other hand each master node can provide external services to read and write services, greatly improving the response capacity of the cluster.
Redis stand-alone memory size is limited problem, introducing persistence and master-slave replication has mentioned; for example, if the stand-alone memory too, bgsave and bgrewriteaof the fork operation may lead to blockage of the main process, if possible, the main switch from the host environment resulting in a long time from the node can not provide services, the entire amount of the replication phase of the master node may copy buffer overflow .......
2, high availability: clustering support automatic failover from the primary transfer and copy the master node (Sentinel and similar); when any node fails, the cluster can still provide services.
Build a cluster:
In this section we will build a simple cluster: a total of six nodes 3 from 3 primary. Convenience: all nodes on the same server, in order to distinguish the port number; configuration simple. 3 the master node port number: 7000/7001/7002, the corresponding port number from the node: 8000/8001/8002.
To build a cluster of two ways:
(1) Redis command performed manually, and step through the building;
(2) use Ruby scripts to build. Both build the principle is the same, but Ruby script Redis commands packaged packaging; recommended scripted in practical applications, quick and easy is not prone to error. The following sections describe these two methods.
Build a cluster can be divided into four steps: (1) Start node: The node in a cluster mode started, when the node is independent and does not establish contact; (2) node shake hands: let the separate nodes connected into a network; ( 3) distribution groove: 16384 slot allocated to the master node; (4) specify the master-slave relationship: the master node from the specified node.
The first building: Manual
(1) Start node
start the cluster nodes are still using redis-server command, but requires the use of cluster mode is activated. The following is a profile 7000 nodes (listed only work key node configuration, other configurations (such as opening AOF) can refer to a stand-alone node):
Redis-7000.conf #
Port 7000
cluster-enabled yes
Cluster-config-File "the Node-7000.conf"
logfile "log-7000.log"
dbfilename "dump-7000.rdb"
daemonize yes
one of the cluster-enabled and cluster- config-file is the configuration associated with the cluster.
cluster-enabled yes: Redis instance be divided into single mode (Standalone) mode and a cluster (cluster); cluster-enabled yes trunked mode can be started. Redis instance starts in standalone mode, info server command if you perform, you can find redis_mode as a Standalone
Cluster-config-File: This parameter specifies the location of the cluster configuration file. Each node in operation, maintains a cluster configuration file; whenever cluster information changes (such as changes in the node), all nodes in the cluster will be updated to the latest information on the configuration file; when the node is restarted, re-read the configuration file, access cluster information, you can easily re-added to the cluster. That is, when a node starts in cluster mode Redis, will first find if there is a cluster configuration file, if you start using a configuration file, if not, the initial configuration and save the configuration to a file. Cluster configuration file maintained by Redis nodes without manual modification.
After editing the configuration files
Use redis-server command to start the node:
redis-server-7000.conf Redis
(2) node handshake
after the start node is independent of each other, unaware of the existence of other nodes; handshaking required node, the separate nodes of a network.
Handshake node using cluster meet {ip} {port} command is implemented
, for example, performing cluster meet 192.168.72.128 7001 node 7000, and node 7000 can complete the handshake node 7001; note that the LAN ip ip used instead localhost or 127.0.0.1 ,
to a node or client on other machines can access
the same token, the use of cluster meet command 7000 nodes, all nodes can be added to the cluster nodes complete the handshake:
1 cluster meet 192.168.72.128 7002
2 cluster meet 192.168. 8000 72.128
. 3 cluster 8001 Meet 192.168.72.128
. 4 cluster Meet 192.168.72.128 8002
(. 3) distribution groove
in Redis cluster, data partitioning to achieve by means of the groove, the principles described will be specifically described. Clusters have 16,384 slots, slot is the basic unit of data management and migration. When the database 16384 slots are assigned node cluster is on-line state (OK); if there is a slot is not allocated to any node in the cluster offline state (fail).
Use distribution groove cluster addslots command, the following command all slots have been allocated (No. 0-16383):
7000 Redis -p-CLI. 1 Cluster addslots {0..5461}
2-CLI Redis Cluster addslots -p {7001} 5462..10922
. 3 Redis -p-CLI Cluster addslots {7002} 10923..16383
(. 4) Specify the main from the relationship between
the specified cluster master-slave relationship is no longer used slaveof command, but the use of cluster replicate command; parameter node id.
After obtaining several id node by the master node cluster nodes, the following command specified for each node from the master node:
. 1 Redis -p-CLI replicate be816eba968bc16c884b963d768c945e86ac51ae 8000 Cluster
2 Redis -p-CLI 8001 Cluster replicate 788b361563acb175ce8232569347812a12f1fdb4
. 3-CLI Redis -p 8002 cluster replicate a26f1624a3da3e5197dde267de683d61bb2dcbf1
./redis-trib.rb create --replicas 1 192.168.72.128:7000 192.168.72.128:7001 192.168.72.128:7002 192.168.72.128:8000 192.168.72.128:8001 192.168.72.128:8002
Wherein: - replicas = 1 means that each node has a master slave node; a plurality of behind {ip: port} represents node address do foregoing shots node, the node from behind. When using redis-trib.rb cluster structures, the requesting node can not contain any data and grooves.
After the process of creating commands, scripts will give you create a cluster plan; plan, including what is the primary node, which is from the node, as well as how to allocate slots
Cluster expansion:
1. Telescopic clusters
often necessary to stretch the cluster, the access operation such as the expansion amount increases practice. Redis cluster can be achieved without affecting the telescopic external services; retractable core migration grooves: groove modify a correspondence relationship with the node, to achieve the groove (i.e., data) between the mobile node. For example, if three slots are evenly distributed in the node in the cluster, when a node is added, it is necessary to come up with a portion of each groove 3 from the node to the new node, thereby achieving a uniform distribution groove 4 nodes.
Add a node,
is assumed to increase the 7003 and the 8003 nodes, where 8003 is the 7003 from the node; the following steps:
(1) Start node: method, see the cluster structures
(2) node handshaking: use cluster meet command, but it is recommended in a production environment redis-trib.rb use of add-node tool, the principle is the cluster meet, but it will first check whether the new node has joined the cluster or there is other data, to avoid adding to the confusion brought about after a cluster.
The Add-redis-trib.rb. 1 192.168.72.128:7003 192.168.72.128 Node 7000
2 redis-trib.rb the Add-Node 192.168.72.128:8003 192.168.72.128 7000
(. 3) Migration grooves: the recommended redis-trib.rb reshard tool to achieve. reshard a high degree of automation, need only enter reshard ip redis-trib.rb: port ( ip and port may be any node in the cluster), follow the prompts and enter the following information, will automatically migrate groove:
○ be migrated groove number: 16384 grooves divided equally among 4 nodes, each groove 4096, and therefore the number of slots to be migrated to 4096
○ target node id: id 7003 nodes
Source node id ○: id 7000/7001/7002 node
(4) specify the master-slave relationship: building a cluster method, see
reducing nodes,
assuming node 7000/8000 to offline, can be divided into two steps:
(1) migration groove: use reshard uniformly migrate to the groove 7000 7001/7002/7003 node node
(2) offline node: using redis-trib.rb del-node tool; should then offline offline node from the master node, because if the main first off the assembly line node, the node will be directed from other master nodes, resulting in unnecessary replication of the full amount.
1 redis-trib.rb del-node 192.168.72.128:7001 { the node id 8000}
2 Redis-del-trib.rb 192.168.72.128:7001 Node id} {7000 nodes
2. failover
cluster only achieved a failover of the master node; from the node failure will only be offline, will not fail. Therefore, when using clusters, it should be used with caution to read and write separation technique, because reading from node failure will cause service is unavailable, the availability of deterioration.
Here the details are not the details of failover, only the important matters Note:
the number of nodes: In a failover stage, need to vote by the master node from which node becomes the new primary node is selected; from node electoral votes needed to win is N / 2 + 1; wherein N number of master node (including the master node failure), the failure of the primary node actually not vote. Thus order to successfully selected from a node, the cluster master node requires at least three (and deployed on different physical machines) when a fault occurs.
Failover time: from the master node fails to complete the transfer, the time required consumed mainly subjective offline recognition, offline subjective propagation delay election several links; parameters related to the specific time cluster-node-timeout, Generally He said:
failover time (ms) ≤ 1.5 * cluster-node- timeout + 1000
default cluster-node-timeout value 15000ms (15s), and therefore will failover time order of 20s
3. The method of limiting and coping clusters
because the cluster data are located in different nodes, causing some limited functions, including:
(. 1) Bulk key operations are limited: for example mget, mset operation, only when the operation key are located in a time slot, can take place. To solve this problem, an idea is the client information record of the key groove, each execution mget / mset specific slot; Another idea is to use the Hash Tag, will be described in the next section.
(2) keys / flushall other operations: keys / flushall operations may be performed in any other node, but the results only for the current node, for example, the operation returns only the keys of all the keys of the current node. To solve this problem, you can use cluster nodes to obtain information about all the nodes on the client and perform keys / flushall and other operations in which all the primary node.
(3) Transaction / Lua script: Cluster Support Services and Lua scripts, but only if the key must be involved in the same node. Hash Tag can solve the problem.
(4) Database: Redis single node may support a database 16, only support a trunked mode, i.e. db0.
(5) Copy structure: Only one replicated structure, does not support nested.
The Hash the Tag 4.
the Hash the Tag principle: when a key containing the {}, the hash does not make the whole key, and comprises only strings do {} hash.
Hash Tag allows different key have the same hash value, so as to distribute in the same slot; so different key for batch operation (mget / mset, etc.), as well as transaction, Lua script and so can support. Hash Tag but can be a problem of uneven distribution of data, then you need to: (1) adjusting the number of slots in different nodes, so that data distribution as even as possible; (2) avoid the use of hot Hash Tag data, the distribution does not cause the request to They are.
Here is an example of using the Hash Tag; product by adding Hash Tag, all products can be placed in the same slot, easy to operate.
The parameter optimization
cluster_node_timeout
cluster_node_timeout initial parameters have been described earlier; its default value is 15s, effects include:
(1) affect the choice of the PING message receiving node: the higher the value the greater the delay tolerance, the less the selected receiving node , the bandwidth can be reduced, but will reduce the rate of convergence; should be adjusted according to the application requirements and bandwidth.
(2) Effect of time and fail determination: the larger the value, the more difficult misjudgment, but the longer the time-consuming to complete the transfer; should be adjusted according to application requirements and network conditions.
cluster-require-full-coverage
mentioned earlier, only when all slots have been allocated 16384, to the cluster line. This is done to ensure the integrity of the cluster, but also brought new problems: When the primary node fails and failover has not been completed, the original master node slots not in any node in the cluster is in the offline state at this time will not be in response to client requests.
cluster-require-full-coverage can change the parameter settings: if set to no, when the groove is not fully allocated, cluster still more lines. The default parameter value is yes, if the application high availability requirements, can be modified to no, but needs its own to ensure that all allocated slots.
Redis-trib.rb 6.
Redis-trib.rb provides a number of utilities: create a cluster, increase or decrease the node, slot migration, integrity checking, data re-balance; you can view the details by the help command. If you can use redis-trib.rb tools are used in practice as much as possible, not only convenient, but also can greatly reduce the probability of errors.
Cache penetrate
cache penetration, refers to certain data query a database does not exist. Normal use caching process generally, data query to cache queries, if there is no key or key has expired, then query the database, and the query to the object into the cache. If the database query object is empty, the cache is not putting them in.
Solution: If an object from the database query is empty, but also into the cache, the cache expiration is set just a short time, for example, set to 60 seconds, does not exceed the maximum 5min
Avalanche Cache
Cache avalanche, refers to a certain period of time, centralized cache expired.
One of the causes of avalanches, as in the time of writing, going to didodecyl zero, will soon usher in a wave of panic buying, this wave of commodity time are concentrated into the cache, the cache assumptions one hour . So that by the time one o'clock in the morning, the cache on these commodities have expired. And for this article can access the query, both fell on the database for the database, it will produce periodic pressure peaks.
Solution: generally take a different classification of goods, cache different periods. In the same category of merchandise, plus a random factor. This can disperse as cache expiration time, and, popular categories of merchandise cache longer time, popular categories of goods shorter buffer time, can save cache service resource
cache breakdown
cache breakdown, it refers to a very key hot, non-stop carrying large concurrency, concurrent large centralized access to this one point, when this key moment in failure, complicated by ongoing large cache wore out, direct requests to the database, as drilling on a barrier a hole.
Greatly do electricity business project, these goods become "explosion models."
In fact, this burst models the most difficult to put pressure on the crush of the database server. The company did not achieve this level of a few. Therefore, the pragmatism of small series of main commodities are ready early, so that the cache never expires. Even some of the goods they became fermentation explosion models, but also directly to never expire just fine. mutex key mutex