What is Redis
Introduction to the official explanation: Redis is an open source project based on BSD, is the structure of a data storage in a system memory, you can use it as a database, caching, and messaging middleware to use.
While supporting strings, lists, hashes, sets, sorted sets, bitmaps, hyperloglogs and geospatial indexes and other data types.
It also has built-in replication, lua script, LRU, affairs, etc., to achieve high availability through redis sentinel, to achieve the automatic slicing through redis cluster. As well as transactional, publish / subscribe, automatic failover, and so on.
In summary, Redis provides a feature-rich, first met may feel confused, why these features are used? What problems have been resolved? Under what circumstances will use the corresponding function? Then the following from scratch, the next step by step evolution to a rough explanation.
1 scratch
The initial demand is very simple, we have provided a list of hot news api: http: //api.xxx.com/hot-news,api consumers complained that each request should be about 2 seconds to return results.
Then we began to look at how to improve the performance of consumer perception api, and soon the most simple and crude first program came out: In response to HTTP-based API plus the cache control cache-control: max-age = 600, that is, so that consumers can cache the response for ten minutes.
If api consumer response if the effective use of the cache control information, can be effectively improved (within 10 minutes) of its perceived performance. But there are two drawbacks: the first is to take effect within 10 minutes of the cache, api consumers may get the old data; the second is if the client api ignore cache direct access to the API still needs two seconds, a temporary na is not a cure.
2 cache memory based on the unit
In order to solve the problems still need to call the API 2 seconds, after investigation, mainly due to the use of SQL get hot news consumed during the nearly two seconds, Ever since, we have thought of a solution for simple and crude, that is results of SQL queries directly in the cache memory of the current api server (set the cache valid time of 1 minute).
Subsequent requests within one minute directly read cache, no longer it takes 2 seconds to perform the SQL. If this time api received 100 requests per second, then one minute is 6000, which is only 2 seconds before the congestion requests takes over 2 seconds, all subsequent requests of 58 seconds can be done even if the response , without having to wait two seconds.
Other API little friends find it is a good idea, and soon we found the API server memory to the full.
3 of Redis server
When the API server cache memory are filled, we had to find another solution for the want. The most direct idea is we put these caches are thrown into it on a dedicated server, much of it to memory configurations. Then we locked in redis. . .
As for how to configure the deployment redis not explained here, redis official detailed description. Then we were using a separate server as Redis server, server memory pressure API solved.
3.1 persistence (Persistence)
Redis server in a single month there are always a few days in a bad mood, bad mood strike, causing all of the cache is lost (redis data is stored in the memory of thing). Although the Redis server back on line, but because the memory of loss of data, resulting in a cache avalanche, pressure API servers and databases or suddenly come up.
So this time Redis persistence function comes in handy, you can ease the impact of cache avalanche brings. redis persistence refers to the redis will write data in memory to the hard disk, load the data when redis restart to maximize the impact of reducing cache misses caused.
3.2 Sentinel (Sentinel) and replication (Replication)
Redis server without warning strike is a hassle. How do you do? The answer: a backup, you hung up on it. So how do you know a station redis server hung up, how to switch, how to ensure that the backup machine is a complete backup of the original server it? This time we need to Sentinel and Replication played.
Redis Sentinel can manage multiple servers, it provides monitoring, alerts, and automatic failover functionality; Replication is responsible for letting a Redis server can be equipped with multiple backup servers. Redis also use these two features to ensure high availability of Redis. In addition, Sentinel is a function of the use Redis publish and subscribe functions.
3.3 Cluster (Cluster)
Single server resources are always capped, CPU resources, and IO resources we can copy from the main, separate read and write, and the pressure part of the CPU is transferred to the IO from the server. But how do memory resources, master-slave mode do is back up the same data, and can not laterally expanded memory; the memory of a single machine can only be increased treatment, but there is always a limit.
So we need a solution that allows us to scale. The ultimate goal is both to each server is only responsible for part of it, so that all these servers constitute a whole, the outside world consumers, this group of distributed servers like as a centralized server (before REST interpretation of the blog explained the difference in distributed network-based: web-based application architecture).
Redis distributed before the official program out there twemproxy and codis two options, both schemes are dependent on the whole proxy for distributed, that does not concern itself redis distributed things, but referred twemproxy and codis to be responsible.
The cluster program redis official is given the distributed part of this thing done every redis server so they no longer require additional components can be distributed independently of the completion of the requirements. Here we do not care slightly superior of these programs, we look at distributed here in the end is to deal with those things? That is processed and distributed twemproxy codis independent processing of this part of the cluster logic and integrated into this part of the logic redis services in the end in solving any problems?
As we said before, a distributed service in the outside world is like as a centralized service. To do this then faced with a problem to be solved: both increase or decrease the number of servers in a distributed service, the customer service end of this consumption should be no sense in terms of; then it means that the client you can not penetrate distributed services, tying themselves to death one server platform up, because if so, you'll no longer be able to add a server, can not fault the replacement.
There are two ways to solve this problem:
The first most direct way that I added a middle layer to isolate this particular dependence, namely twemproxy methods used, so that all clients can only use it to consume redsi service through which to isolate this dependence ( but you will find twermproxy will become a single point), each redis server in this case are independent, they do not know each other's existence among each other;
The second way is to let the server know redis each other's existence, to guide the client through the redirection mechanism to complete the operation they need, such as the client is linked to the one redis server, that I want to perform this operation, redis server discovery they can not complete this operation, you'll be able to complete this operation information to the server to the client, allowing the client to request another server, this time you will find that every redis server needs to maintain a complete distribution an information server information, or to let the client know how it went the other servers to which the client wants to perform an operation of it.
This explains the large segment above so much, I wonder whether it is not found the first way or the second way, there is a common thing there is, and that is a distributed information service for all servers and services they can provide . The information to be present in any case, except that the first part of this information is a way to manage alone, use this information to coordinate multiple independent redis server back end; the second way is to let each redis server hold this information, know each other's existence to each other, and the first to achieve the same objective way, the advantage is no longer a need for additional components to process this part of the thing.
Specific implementation details Redis Cluster of is the use of the concept of Hash grooves, i.e., pre-dispensed 16,384 grooves: the client by Key for CRC16 (key)% 16384 operation to obtain a corresponding groove which one; at redis server is each server is responsible for a portion of the slot, when a new server is added or removed, before we migrate these slots and their corresponding data, and each server are in possession of information on the full tank and its corresponding server, which so that the server can request the client redirection processing.
4 client Redis
The third section focuses on the upper end of the service is Redis evolutionary step, he explains how Redis from a stand-alone service, evolved into a highly available, decentralized, distributed storage system. This section is concerned at the client can consume redis service.
4.1 Data Types
redis support for rich data types, from the most basic to complex string of commonly used data structures are supported:
- string: the basic data types, binary safe string, maximum 512M.
- list: a list of the string holding the order according to order of addition.
- set: a set of unordered string, repeat element exists.
- sorted set: a collection of sorted strings.
- hash: key-value pairs that one kind.
- bitmap: A more detailed operation in bit units.
- hyperloglog: probability-based data structure.
The large number of data types, primarily to support the needs of a variety of scenarios, of course, each type has a different time complexity. In fact, before these complex data structures equivalent to introduce me to the architectural style of network-based applications in the "interpretation REST" This series of blog Remote Data Access (Remote Data Access = RDA) of concrete implementation, namely through the implementation of a set on the server standard operation command, the result obtained between the server wants to set the reduced, thereby simplifying the use of the client, it can also improve network performance. For example, if there is no such list data structure, you can only save the list into a string, clients get a complete list, and then complete the operation submitted to redis, will have a lot of waste.
4.2 Transaction
The above data types, each data type has its own command to operate, in many cases we need to execute a command more than once, and at the same time it needs to succeed or fail. redis transaction support is derived from this part of the demand, i.e., the ability to support a plurality of commands performed once in sequence, and to ensure its atomicity.
4.3 Lua script
On the basis of the transaction, if we need to perform a one-time service at the end of a more complex operation (contains some logic judgment), then lua can rafts in handy (for example, in obtaining a certain cache when it expires at the same time extend the time ). redis ensure lua script atomicity, under certain scenarios, it can substitute transaction-related commands redis provided. Corresponds to the specific implementation of the architectural style describes the network application based remote evaluation (Remote Evluation = REV) of.
4.4 Pipeline
Because, by default every time when TCP-based connection can only execute a command when connected redis client and server. It allows the use of a pipe is connected to a plurality of command processing, so as to save some of the overhead tcp connection. Difference is that the pipe conduit and the transaction is to save the cost of communication, it does not guarantee atomicity.
4.5 Distributed Lock
The official recommended Redlock algorithm that uses the string type, the time to lock in a specific key, and then set a random value; cancel lock when using lua script with the implementation of access to first compare and then delete the key. Specific command is as follows:
SET resource_name my_random_value NX PX 30000
if redis.call("get",KEYS[1]) == ARGV[1] then
return redis.call("del",KEYS[1])
else
return 0
end
to sum up
Benpian focus on to explain the various functions under redis from the abstract level and the purpose of its existence, and not care what the specific details Yes. So it can focus on solving the problem, according to abstract concept level can make us choose a more suitable solution in specific scene, and not confined to the technical details.