Redis
Redis five data types and message subscription
String command
String is the most commonly used type of data, common key / value storage can be classified as such
1：set/get
Set the value of type String key value corresponding to
Obtaining a value corresponding key
2: mget
bulk values of a plurality of key, if there may be no return nil
3：incr && incrby
incr value corresponding to the key operation performed jerk, and returns the new value; adding specified value incrby
4：incr && incrby
incr value corresponding to the key operation performed jerk, and returns the new value; adding specified value incrby
5：setnx
String type of the set value corresponding to the value key, if the key already exists returns 0
6：setex
String type of the set value corresponding to the value key, and set the period
Other commands
getrange get key corresponding to the value of the substring
mset batch set multiple key values, and if successful, means that all values are set, otherwise it returns 0 means no value is set msetnx, with mset, setting does not exist, will not overwrite some key
GetSet provided key value, and returns the old key value append: adding value to the specified key character string, and returns the new string length
Hash command
hash table is a mapping between a field of type String and value (double-Hash)
The data type redis Hash key (hash table name) corresponding to the actual value of the internal storage structure of a HashMap
Hash is particularly suitable for storing objects
With respect to each attribute of an object is stored as a String type, the entire object stored in the Hash type consume less memory.
Members of the stored data is stored is less zipmap, when increasing the number of members will be automatically converted into a real HashMap, this time for the encoding ht
The use of scenarios: If using an object to store user information, product information, order information, and so on.
hset: Set key corresponding to the field value in the HashMap
hget: acquiring key corresponding to the field value in the HashMap
hgetall: Get all of the field corresponding to the key value in the HashMap
hlen: Returns the number key corresponding to the field the HashMap
List of command
lpush: Add a key element in the list corresponding to the head
lrange: Get a list of key elements corresponding to the specified index range, retrieves all elements -1
lpop: Remove a key element from the corresponding end of the list, and returns the element
rpush: Add a key element in the corresponding end of the list
rpop: Remove a key element from the corresponding end of the list, and returns the element
Set command
sadd: Add a key element in the corresponding set of
smembers: get key corresponding to the set of all elements
A delete key and returns a random element corresponding to the set: spop
suion: request set corresponding to the given key and set
sinter: seeking the given key set corresponding to the intersection
SortSet command
Basis set in increasing order of score, and then sorted according to score
zadd: Add a key element in the corresponding zset
zrange: Get zset key corresponding element in the specified range, retrieves all elements -1
zrem: delete a key element in the corresponding zset
zrangebyscore: returns an ordered set of key, the list of elements to specify a range of scores, rankings use
zrank: Returns the key corresponding to the specified zset member rankings. Score by which the member value is incremented (small to large); bottom ranking to 0, that is, the smallest member of score ranking value is 0, the use of rankings
set by hashmap storage, key corresponding to the set of elements, value is an empty object sortset is how to store and sort of realize it, hashmap storage, but also add a layer jump table jump table: the equivalent of a doubly linked list, add it on the basis of travel larger than the current element hoplinks
Redis is an open source (BSD license), the data structure stored in the memory system that can be used as a database, cache and messaging middleware. It supports multiple types of data structures, such as a string (strings), Hash (of the hashes), lists (Lists), set (sets), an ordered set (sorted sets) and the like.
Redis news release subscription
Role: Subscribe to publish similar information pipeline, used to decouple messages between systems, similar mq, rabbitmq, rocketmq, kafka, activemq major news publishers and news subscribers. For example used: Orders successful payment, plus membership system integration, operating deducting money purse, delivery systems (issued goods)
PUBLISH: the message to send information to the designated channel channel. Returns the number of client receives the message
SUBSCRIBE: Subscribe to information specified channel
UNSUBSCRIBE: unsubscribe specified channel, if not specified, unsubscribe from all channels.
? Subscribe to news releases and mq contrast Redis answer: redis publish subscribe function is relatively weak but relatively lightweight, mq message persistence, data reliability is poor, no back-office functions can msgId, msgKey query message
Redis transaction mechanism
MULTI and EXEC command
MULTI to start a transaction, and then multiple commands into teams to the transaction, the transaction is triggered by the last EXEC command, together with all the commands in the transaction
DISCARD command
DISCARD command is used to cancel a transaction, it is clear the entire transaction queue of the client, then the client adjusted back from the transaction state non-transactional state, and returns the string OK to the client, indicating that the transaction has been canceled
WATCH command
WATCH command before the transaction begins to monitor any number of keys: When calling EXEC command to execute a transaction, if any one
Monitored other client key has been modified, then the entire transaction is no longer executed, direct return failure
Compare redis affairs with traditional relational transaction
Atomicity (Atomicity)
Redis execute a single command is atomic, but Redis without adding any mechanism for maintaining atomicity in transaction, so the implementation of Redis transaction is not atomic. If a transaction queue all commands have been successfully executed, said the success of this transaction executed
Consistency (Consistency)
The wrong team
During the command into the team, if the client sends the wrong command to the server, not the number of parameters such as commands, and so on, then the server to the client returns an error message, and the client transaction state is set REDIS_DIRTY_EXEC.
Execution error
If a command error occurs during execution of a transaction, for example, to perform a different type of key to the wrong operation, the error will only Redis included in the outcome of the transaction, this transaction will not cause disruption or failure of the whole, not will affect the outcome of the transaction has been executed command, it will not affect the transaction following the command to be executed, so the consistency of its transactions are not affected
Isolation (Isolation)
WATCH command before the transaction begins to monitor any number of keys: When calling EXEC command to execute a transaction, if any
Italy is a key has been monitored by another client changed, then the entire transaction is no longer executed, direct return failure
Persistent (Durability)
Because the transaction but is played with a set of Redis command queue wrapped, it did not provide any additional persistent sexual function, so persistent persistent mode used by the Redis transaction decisions
Endurance of
RDB persistence means that within a specified time interval memory snapshot of the data set is written to disk, the actual operation is fork a child process, the first set of data written to a temporary file, write successfully, and then replace the file before , binary compressed storage.
AOF persisted in the form of log records processed by the server every write, delete, query operation is not recorded, the recorded text, you can open the file to see the detailed operating record.
Both advantages and disadvantages
What are the advantages RDB exist?
1) Once this manner, then your entire Redis database will contain only one file, which for file backup is perfect. For example, you might intend to archive every hour last 24 hours of data, but also archived once in the last 30 days of data every day. With this backup strategy, once the system has failed catastrophically, we can very easily be restored.
2) For disaster recovery purposes, RDB is a very good choice. Because we can very easily be a separate file compression and then transferred to other storage media.
3) The performance is maximized. For service of process Redis, at the beginning of persistence, it is the only need to do is fork out the child, and then after the completion of these persistent work by the child, so that you can greatly avoid the process of an IO operation of the service.
4) Compared to the AOF mechanism, if the data set is large, start RDB efficiency will be higher.
RDB and what weaknesses exist?
1) If you want to guarantee high availability of data, that is, the maximum to avoid loss of data, then the RDB would not be a good choice. Because once the system downtime phenomenon before the timing of persistence, had not had time to write data on the disk will be lost.
2) Since the RDB through the fork a child process to assist with data persistence work, so if and when the data set is large, it may cause the entire server to stop serving hundreds of milliseconds, or even one second.
AOF advantage, what does?
1). This mechanism can result in higher data security, namely data persistence. Redis provides 3 in synchronization strategy, that sync every second, every modification and synchronization are not synchronized. In fact, asynchronous synchronization is complete per second, its efficiency is very high, the difference is that once the system downtime phenomenon, then within seconds the modified data will be lost. And every modify the synchronization, we can be regarded as synchronization persistence that occurs every time data changes will be immediately recorded to disk. It is foreseeable that in this way is the lowest in efficiency. As for non-synchronous, needless to say, I think we can correct understanding of it.
2). Since the mechanism writes to the log file append mode is used, even if the phenomenon of downtime during the writing process, without disturbing the contents of the log file already exists. However, if we write this operation is only half the data appeared system crash, do not worry, before the next startup Redis, we can to help us solve the problem of data consistency by redis-check-aof tool.
3) If the log is too large, Redis can automatically enable rewrite mechanism. That Redis to append mode will keep the modified data is written to the old disk file, while Redis also creates a new file to record what has changed on command is executed during this period. Therefore, when switching is performed rewrite data security can be guaranteed better.
4). AOF format comprising a clear, comprehensible log file for recording all the changes. In fact, we can complete the reconstruction data through the file.
AOF disadvantages, what does?
1) For the same number of data sets, AOF is generally larger than RDB document file. RDB speed when recovering large data sets faster than the speed of recovery of AOF.
2). Depending on the synchronization policy, AOF on the operating efficiency tends to be slower than in RDB. In short, the efficiency of synchronous per second strategy is relatively high, disable synchronization efficiency and RDB policy as efficient.
When both selection criteria, is to see the system is willing to sacrifice some performance in exchange for higher cache coherency (aof), I was willing to write frequent operation, do not enable the backup in exchange for higher performance, time to be run manually save the , do a backup (rdb). This is even more rdb some eventually consistent mean.
redisTemplate api
opsForValue operation String, Key, Value, comprising a set operating opsForSet expired key, setBit bit manipulation
opsForHash operation the hash
opsForZset operation SortSet
opsForList operation queue list
opsForHash operation hash opsForZset operation list queue operation SortSet opsForList
Cache avalanche
What is the cache avalanche? Do you have any solutions to prevent caching avalanche?
If the cache is concentrated over a period of time has expired, a lot of cache penetration occurs, all queries fall on the database, resulting in a cache avalanche. Since the original cache invalidation, the new cache yet reached all requests for access to the cache during which should go to query the database, while the database enormous pressure on the CPU and memory, can cause serious database downtime
Do you have any solutions to prevent caching avalanche?
1: lock queue
key: whiltList value: 1000w a uid designated setNx whiltList value nullValue mutex mutex to solve, Redis of SETNX to set a mutex key, when the operation returns successfully, then the operation load db and back in the cache; otherwise, retry get the entire cache method
2: Data preheat
After preheating the cache line is on the system, the relevant data is directly loaded into the cache buffer system. This can be avoided when the user requests, first query the database, and then the data cache problem! Users to directly query cache data previously been preheated! Cache reload mechanism can advance to update the cache, then the impending large concurrent access pre-loaded manually trigger different cache key
3: double caching policy
The original cache C1, C2 is a copy of the cache, when C1 fails, you can visit C2, C1 cache invalidation time is set short-term, C2 is set for long-term
4: regularly updated caching policy
Cache invalidation less demanding, start the initialization container loading, updated or removed using the timer task cache
Set different expiry time for the cache invalidation point in time as uniform as possible
Cache penetration
What is the cache penetrate? Do you have any solutions to prevent caching penetration?
Cache penetration refers to the user query data, not in a database, naturally there will not be in the cache. This results in the user's query when the corresponding key is not found in the cache of value, every time to re-query the database again, and then return empty (the equivalent of useless conducted two inquiries). Such requests will bypass the cache Direct Access database
Do you have any solutions to prevent caching penetration?
1: The Bloom filter BloomFilter
All possible data hash to a sufficiently large bitmap in a certain absence of this data will be
bitmap interception off, thus avoiding queries pressure on the underlying storage system
2: Cache null
If a query returns the data is empty (data not whether, or system failure exists) we still see the empty cache results, but its expiration time will be very short, no longer than five minutes. Through this direct deposit set default values to the cache, so the second time to get there is value in the buffer, but will not continue to access the database
Cache benefits and costs
Cache payoff:
1: acceleration speed read and write cache read and write speeds: CPU L1 / L2 / L3 Cache, Linux page Cache acceleration drives the reader, the browser cache, the cache database results Ehcache
2: back-end servers to reduce the load reduces the load the rear end through the front buffer: reducing the cost of service end use Redis cache brings MySQL rear end loads or the like:
1: inconsistent data caching and data layers have a window of time inconsistency, and update policies related
2: The original code maintenance costs can be achieved only need to write MySQL function, but after adding the cache going to maintain cached data, increasing code complexity. The potential risk in the cache memory heap overflow affect user processes, such as ehCache, loadingCache
Within the heap cache and remote server cache redis selection:
A heap buffer within the general better performance, remote cache needs socket transport
Wherever possible, remote user level cache cache
Wherever possible, remote large data cache, the principle of serving node
redis two ways to use as a database and as a memory cache
redis used as a database of what advantages and disadvantages
Advantages: Scheme no constraint, relatively easy to change the data structure, the data type determines the outset, compression capability, high performance, 100,000 / qps
Disadvantages: no index, no foreign keys, lack int / date basic data types, many conditions need to develop low indirect efficiency inline set (sinter, zinterstore) and a connection, poor maintainability
redis used as cache, with two programs used by the database
jedis integration scheme set key, value [ "11", "22"] to query a first layer in the buffer, if the data is returned directly to give the second layer in a database query, and flush the cache next time the query [ " 33, "44"] as mybatis / hibernate secondary cache scheme, a cache: sqlSession, caching process, a single link effective
Transaction isolation mechanism
Syntax: set global transaction isolation level read uncommitted ;
种类:read uncommitted、read committed、repeatable read、serializable
