1. What is redis?
Redis is a memory-based high-performance key-value database.
2.Reids features
Is the Key-Value Redis essentially a type of memory database, the memcached like, operates the entire database in memory to load all of them, the asynchronous operation by periodically flush the database data to be stored on the hard disk. Because it is pure memory operation, Redis performance is very good, can handle more than 100,000 times per second read and write operations, it is known to the fastest performance of Key-Value DB.
Redis not only excellence of performance, Redis greatest charm is to support a variety of data structures stored, in addition to the maximum limit of a single value is 1GB, unlike memcached data can only be saved 1MB, so Redis can be used for many useful function, for example to do with his List FIFO doubly linked list to achieve a lightweight, high-performance message queuing service, with his Set can do high-performance tag systems. In addition Redis can also set expire time of deposit of the Key-Value, it can also be used as a function enhanced version of memcached to use.
The main disadvantage of Redis database capacity by limiting physical memory, reading and writing massive data can not be used in high-performance, and therefore the appropriate scene Redis mainly confined to the small amount of data and high-performance computing operations.
3. Use redis What are the benefits?
(1) faster because the data stored in memory, similar to HashMap, HashMap advantage is to find the time and complexity of the operation is O (1)
(2) support for rich data types, support string, list, set, sorted set , hash
. 1 ) String common commands: SET / GET / DECR / incr / mget the like; scenarios: String is the most commonly used type of data, Common Key / value storage can be classified as such; implementations: the internal String redis the default is stored in a string to be referenced redisObject, when faced incr, decr other operations will turn into numerical calculation, encoding field redisObject case of an int. 2 ) Hash commonly used commands: hget / HSET / hgetall etc. Scenario: We want to store a user information object data, including user ID, user name, age and date of birth, by user ID, we want to get the user's name or age, or date of birth ; implementation: Redis of Hash Value is actually stored internally as a HashMap, and provides direct access to members of the Map interface. As shown, Key is the user ID, value is a Map. The Map is a key attribute of members, value is the attribute value. Such data can be accessed and modified directly inside the Map Key (Redis Map key in said interior of field), i.e. by a key (user ID) +Field (Properties tab) can attribute data corresponding to the operation. The current implementation of HashMap in two ways: When a member HashMap relatively little time Redis to save memory will adopt a similar approach to the one-dimensional array of compact storage without using real HashMap structure, when encoding redisObject the corresponding value for the zipmap, when increasing the number of members it will be automatically converted into a real HashMap, this time for the encoding ht. hash 3 ) List commonly used commands: LPUSH / RPUSH / lpop / RPOP / lrange etc; scenarios: Redis list of scenarios is very large, Redis is also one of the most important data structures, such as Watchlist, a list of twitter fans and so can with the Redis list structure to achieve; implementation: Redis list is implemented as a doubly linked list that can support the reverse lookup and traversal, more convenient operation, but bring some extra memory overhead, a lot of internal implementation Redis, including the transmission buffer queue data structure, etc. are also used. 4 ) the Set commonly used commands: Sadd / SPOP / smembers / SUNION etc; scenarios: Redis set of external functions provided with the list function is similar to a list of special is that is can be set to automatically de-duplication, when you need to store a when the list data, do not want to duplicate data set is a good choice and set provides important interfaces determine whether a member within a set collection, this list is not offered; implementation: setThe internal implementation is always a value to null HashMap, is in fact a calculated hash way to quickly de-duplication, which is set to provide the members determine whether a reason in the collection. . 5 ) the Set the Sorted common commands: Zadd / Z Range The / zrem / zcard the like; scenarios: Scene and using similar set Redis sorted set, the difference is not self-ordering the set, and can provide the sorted set a priority by a user additional ( score) for the parameters to sort the members, and is inserted into a sorted, i.e. automatic sorting. When you need an ordered list of non-repetition and collection, you can select the sorted set of data structures, such as twitter's public timeline can be stored up to post time as the score, time is automatically sorted when such acquisition. Implementation: Redis sorted set of internal use HashMap and jump table (SkipList) to ensure the orderly and store data, put HashMap are members of the maps to score, and jump table is stored in all the members, is the sort HashMap stored in the score, using a jump table structure can be obtained relatively high search efficiency, and relatively simple in implementation.
(3) support services, operations are atomic, so-called atomic changes to the data that is either all executed or not executed all
post can be used for caching, message, press the key to set the expiration time, expires: (4) rich feature will be automatically deleted
4.redis memcached What are the advantages compared to?
All values (1) memcached are simple string, as an alternative Redis support richer data type
(2) redis is much faster than memcached (3) redis can persist its data
Redis difference 5.Memcache and have what?
(1), the data storage Memecache all present in the memory, will hang power failure, data can not exceed the memory size. Redis has a part on the hard disk, this will ensure persistent data.
(2), the data type of support for data types supported Memcache relatively simple. Redis complex data types.
(3) using the underlying protocol for communication between different application model underlying implementations between them and the client is not the same. Redis VM build their own direct mechanism, because most of the system call system function, it will waste some time to move and requests.
6.redis apply to the scene?
Redis most suitable for all data in-momory scenes, such as:
(1), the session cache (Session Cache)
The most common scenario is using Redis session cache (session cache). Redis caching session with than other storage (such as Memcached) has the advantage of: Redis provide persistence.
(2), page caching (FPC)
In addition to the basic session token, Redis also provides a very convenient platform for FPC. Back consistency, even restart Redis instance, because of the persistent disk, users will not see a decrease page loading speed, which is a great improvement, similar to PHP local FPC.
(3), the queue
Reids a big advantage in the field memory storage engine is to provide a list and set operations, which makes Redis can serve as a good platform to use the message queue. Redis used as an operation queue, similar to native language (e.g., Python) push on the list / pop operations.
If you do a quick search for "Redis queues" in Google, you'll be able to find a large number of open source projects, the purpose of these projects is to use Redis create a very good back-end tools to meet the needs of various queues. For example, Celery has a background is to use Redis as a broker, you can go to see from here.
(4), charts / counter
Redis digital up or down in memory operations are implemented very well. Collection (Set) and an ordered set (Sorted Set) also allows us the time to perform these operations become very simple, Redis just provide just these two data structures. So, we need to sort the collection to get the highest ranked 10 users - what we call "user_scores", we just need to execute something like the following:
Of course, this assumes you are doing increasing user score based on your sort. If you want to return the user and the user's score, you need to perform:
ZRANGE user_scores 0 10 WITHSCORES
Agora Games is a good example, implemented in Ruby, its ranking is to use Redis to store data, you can see here.
(5), publish / subscribe
Last (but certainly not least) is Redis publish / subscribe functionality. Publish / Subscribe usage scenario is indeed very much.
7, Redis cache invalidation strategies and primary key failure mechanism
As the cache system must be cleaned regularly invalid data, we need a primary key failure and elimination strategies.
In Redis, among them, survival is key called volatile. When you create a cache, to set a lifetime for a given key, when the key expired (0 lifetime), it may be deleted.
1, some operations on survival time of
survival time can be deleted by using the DEL key to remove the entire command, or by overwriting the original data and GETSET SET command, i.e., value, and using the modified additional key corresponding to the same key and after value to cover the current different survival time data.
For example, the implementation of a key INCR command, for a list LPUSH command, or execute a command HSET hash table, such operations are key itself does not modify the survival time. On the other hand, if you use a RENAME key to be renamed, then the survival time after the key was renamed and renamed as before.
RENAME command Another possibility is to try to survive a key time with another band renamed another_key survival time, when the old another_key (and its survival) will be deleted, then the old key will be renamed another_key, therefore, the new another_key survival time and the same as the original key. Use PERSIST command without delete key, removing key survival time, so that key again become a persistent key.
2, how to update the survival time
can be executed EXPIRE command key has a survival time with new specified survival time will replace the old survival time. Expiration time precision has been controlled within 1ms, failure of the primary key of the time complexity is O (1),
EXPIRE command and with the use of TTL, TTL can view the current key of survival. Success, return 1; when the key does not exist or can not set the survival time key, returns 0.
The maximum cache configuration
in redis, allowing the user to set the maximum memory used
server.maxmemory
default is 0, there is no specified maximum cache, if new data is added, exceeds the maximum memory, you will redis collapse, so be sure to set up. redis memory size of the data set to rise to a certain size and they will implement data elimination strategy.
redis offers six data elimination strategy:
. volatile-lru: selection of the least recently used data out of the set of data set expiration time (server.db [i] .expires) in
. volatile-ttl: selection of data to be expired expiration time has been set out from the data set (server.db [i] .expires) in
. volatile-random: the selection data set expiration time has been eliminated from the data set (server.db [i] .expires) any
. allkeys-lru: selection of the least recently used data out from the data set (server.db [i] .dict) in
. allkeys-random: selecting out data from the data set (server.db [i] .dict) any
. no-enviction (expulsion): prohibits the expulsion data
Note 6 kinds of mechanisms here, volatile and allkeys provides is a set expiration time of the data collection phase-out of data or from the entire set of data out of the data behind the lru, ttl and random three different kinds of elimination strategies, plus one kind of no-enviction never recovered strategy.
Use policy rules:
1, if the data presented power-law distribution, which is the high part of the data access frequency, low part of the data access frequency, use AllKeys-the LRU
2, if the data is rendered equal distribution, that is, all data access frequency are the same, allkeys-random using
three data phase-out strategy:
ttl and random easier to understand, will be relatively simple to achieve. Lru least recently used mainly eliminated strategy will fail sorted by key design time, and then take the key out of the first failure were
8. Why redis need to put all the data into memory?
Redis To achieve the fastest read and write speed data into memory, and by asynchronously writing data to disk. Therefore, fast and redis data persistence features. If you do not put the data in memory, disk I / O speed to seriously affect the performance of redis. In memory getting cheaper today, redis will become increasingly popular.
If you set the maximum memory used, the data has been recorded can not continue to insert a new value after the number reached memory limit.
9 .Redis is single-threaded single-process
redis queuing technologies will become concurrent access serial access, eliminating the overhead of traditional database Serial Control
10 How .redis concurrent competition problem solving?
Redis is single-threaded process model, using concurrent access to the queue model will become serial access. Redis itself does not lock concept, Redis for multiple client connections competition does not exist, but the connection timed out when Jedis Redis clients for concurrent access occurs, data conversion error, blocking, client and close the connection problems that are
Because of the confusion caused by client connections. In this regard there are two solutions:
1. The perspective of the client, for each client in order to ensure a graceful between Redis communication with, for connection pooling, while the client read and write operations using the internal lock Redis synchronized.
2. Server angle, achieved using setnx lock.
Note: For the first, you need to deal with their own synchronization application resources, methods that can be used more popular, you can use synchronized can also use the lock; the second is the need to use Redis setnx command, but note some problems.
11, redis common performance problems and solutions:
1) .Master write memory snapshot, save rdbSave command scheduling function, can block the main thread of the work, when the snapshot is relatively large impact on performance is very large, it will be suspended intermittently, so Master is best not to write a memory snapshot.
2) .Master AOF persistent, if not rewrite AOF documents, the impact of this persistent way on performance is minimal, but AOF files grow, AOF file too much effect on the speed of recovery Master restart. Master best not to do any persistent work, including memory snapshots and AOF log files, especially not enabled to do lasting memory snapshot
Of, if the data is more critical, open a Slave AOF backup data, sync strategy once per second.
3) .Master call BGREWRITEAOF AOF rewrite the file, when AOF rewrite will account for a large amount of CPU and memory resources, resulting in service load is too high, a brief suspension of service phenomenon.
4). Redis replication master from performance problems, in order from the master copy speed and stability of the connection, the Slave and Master preferably in the same LAN.
Learn 12.redis things CAS (check-and-set operation to achieve optimistic locking)?
And, like many other databases, Redis NoSQL databases as also provides transaction mechanisms. In the Redis, MULTI / EXEC / DISCARD / WATCH four command is cornerstone of our affairs. I believe developers have a relational database development experience in terms of the concept is not new, even so, we will briefly list
Redis in
Implementing the features of the transaction:
1) all the commands in the transaction are executed will be serialized in the order, during the execution of the transaction, Redis will not provide any services to requests from other clients, thus ensuring that all the things in order It is performed atoms.
2). And relational database transactions compared to if there is a failure of a command, the command that follows will still continue to be in Redis transaction.
3) We can open a transaction MULTI command, there is a relational database development experience can be understood as a "BEGIN TRANSACTION" statement. After this statement is executed command will be treated as operating within the affairs, and finally we can commit / rollback all operations in the transaction by executing the EXEC / DISCARD command. Both Redis commands can be considered equivalent to a relational database COMMIT / ROLLBACK statement.
4) Prior to the transaction open, if the communication failure between the client and server and network disconnect cause, all subsequent statements to be executed will not be executed server. However, if after a network outage event occurs in the client execution EXEC command, all commands will be executed the transaction server.
5) When using Append-Only mode, Redis all write operations by calling a function in the system the write transaction to write the entire disk in this call. However, if the writing occurs during a system crash, such as downtime due to power failure, then the time perhaps only part of data is written to disk, while the other part of the data has been lost. Redis server will perform a series of necessary consistency check when you restart, once found similar problems, will quit immediately and give appropriate error message. At this point, we must take full advantage of redis-check-aof Redis tool kit provided, which can help us to locate errors inconsistent data, and rolls back the portion of the data has been written. After the repair we can restart Redis server up again.
13.WATCH command and a CAS-based optimistic locking?
In Redis transaction, WATCH command can be used to provide CAS (check-and-set) function. Suppose we pass WATCH commands to monitor multiple Keys before transaction execution, if there is any value in Key has changed after WATCH, EXEC command to execute the transaction will be discarded, returning Null multi-bulk reply to inform the caller Affairs
Execution failed. For example, we once again assume Redis incr not provided in order to complete the atomic increment key, if you want to achieve this, we can only write the code yourself. Pseudocode which follows:
Val MyKey the GET =
Val = Val +. 1
the SET MyKey Val $
above code only in case of a single connection can guarantee execution results are correct, because if there are multiple clients at the same time while performing the sections of the code, then there will be a wrong scene multithreaded programs often appear - race race (race condition). For example, both clients A and B at the same time reading the original value mykey, assuming the value of 10, after which the two clients and average value plus a set back after Redis server, which would lead to mykey the result is 11, rather than what we think of 12. In order to solve similar problems, we need to with the help of WATCH command, see the following code:
WATCH mykey
Val = GET mykey
Val = Val + 1
the MULTI
the SET mykey $ Val
EXEC
and after the code is different, before the new code to obtain the value mykey of WATCH command to monitoring by the key, then turn the set command surrounded in a transaction, so that you can effectively ensure that each connection before executing the EXEC, if the current value of the acquired mykey connection is modified other connected client, the current EXEC command connection will fail. In this way the caller after the return value can be learned whether val is reset successfully.
14. Used Redis distributed lock it, it is what is going on?
Setnx scramble to acquire a lock, then grabbed, and then add a lock to expire expiration prevent lock forget released.
This time the other will tell you, you answer quite well, and then went on to ask if accidental crash before execution expire after setnx process or to restart maintained, then what happens?
This time you have to give feedback surprise: Well, is oh, this lock will never be released. Then you need to get yourself and scratched his head, pretending to think for a moment, as if the next result is that you take the initiative to think out, then answered: I remember a very complex instruction set of parameters, this should be possible and expire at the same time the setnx synthesis of an instruction to use! Then the other side will reveal a smile, and my heart began to recite: press, this kid is not bad.
15. If there are 100 million Redis key, which has a key 10w begins with a fixed known prefix, if all they find out?
Use keys instructions may sweep out a list of the specified pattern key.
Then ask each other: If this redis is providing services to the business line, use keys that instruction will be a problem?
This time you have to answer a key characteristic redis: redis single-threaded. keys instruction causes the thread blocked for some time, online services will be stalled until the instruction is finished, the service can be restored. This time can be used scan command, scan command can extract the key non-blocking list specified pattern, but there will be some duplication of probability, on the client side deduplication can be done once, but the overall time spent directly with the ratio keys commander.
16. Used Redis what to do asynchronous queue, how do you use?
Structure is generally used as a queue list, rpush message production, lpop consumption message. Lpop when no message to a proper sleep will try again.
If the person asking it Can not sleep? There is a list of instructions called blpop, in the absence of information, it will block until the message arrives.
If the person can not produce a consumer asked many times it? Use pub / sub topic subscriber mode, you can achieve the 1: N message queue.
If they ask pub / sub What are the disadvantages? In the case of the consumer off the assembly line, production will be lost messages, message queues have to use professional as rabbitmq and so on.
If the person asking how to achieve redis delay queue? I guess now you want to put on a stick and killed the interviewer if you have got a baseball bat, then, how to ask so detailed. But you very restrained and calm demeanor replied: Use sortedset, take a timestamp as score, the message content as key to call zadd to produce news, consumer access to polling data N seconds before treated with zrangebyscore instruction.
Here, the interviewer has secretly you a thumbs-up. But he did not know at the moment is that you have put up a middle finger, the back of the chair.
17. If a large number of key needs to be set at the same time has expired, the general needs attention?
If a large number of key expiration time set too concentrated, at that point in time expired, redis may momentarily Caton phenomenon. Generally need to add a random value over time, the expiration time such that some of the dispersion.
How do 18.Redis persistence?
bgsave to mirror the full amount of persistence, aof do incremental persistence. Because bgsave will take a long time, not real time, when a lot of downtime can result in loss of data, so it is necessary to use with aof. When redis instance restart, use bgsave persistent file to reconstruct the memory, and then use aof replay with recent operating instructions to cause a state of complete recovery before restarting.
The other asked that if a sudden power failure the machine will happen? Depending on the configuration aof log sync attributes, if performance is not required, at the time of each disk write instruction sync it, they will not lose data. But in the high performance requirements of every sync it is not realistic, generally using a timed sync, such as 1s1 times, this time it will lose most of the data 1s.
The other asked what bgsave is the principle? You are given two words on it, fork and cow. fork means redis to bgsave by creating child processes, cow means that the copy is created on write, the child, the parent and child share the data segment, the parent process continues to provide literacy services to write dirty page data will gradually and sub-process separation off.
19.Pipeline What are the benefits, why should the pipeline?
Multiple IO round-trip time can be reduced to one, provided that no causal correlation between the instruction execution pipeline. Redis-benchmark used for pressure measurement can be found when a peak value of QPS redis important factor is the number of batch instruction pipeline.
20.Redis synchronization mechanism to understand it?
Redis master-slave synchronization may be used, from the synchronization. The first synchronization, the master node do bgsave once, while the recording operation of the subsequent modifications to the memory buffer, after completion rdb be synchronized to the total amount of file copy node, after the completion of copying node accepts rdb image loaded into memory. After loading is complete, and then notify the master node during operation of the modified copy to the node record synchronization reproduction synchronization process is completed.
21. Whether used Redis cluster, the cluster is what principle?
Redis Sentinal focus on high availability, downtime in the master slave automatically promoted to master, continue to provide services.
Cluster redis focus on scalability, while a single redis insufficient memory, memory fragmentation using Cluster.
Reference blog: https://www.cnblogs.com/doit8791/p/8563667.html