"Redis Optimization Strategy"

About Redis general properties

Persistence Strategy in Redis-RDB

1-Requirements description:

The data in Redis is stored in the memory, but the characteristics of the memory are power-off and erased. In order to ensure that the cached data in redis is not lost, the memory data needs to be periodically persisted.

Endurance:将内存数据,写到磁盘中

2-RDB mode

Features:

• RDB mode is the default persistence rule of Redis
• The RDB mode records a snapshot of Redis memory data (only the latest data is kept)
• Regular persistence of RDB mode (time adjustable) may cause data loss.
• RDB mode backup efficiency is the highest.
• RDB mode backup blocking type does not allow other users to operate during backup. Ensure data security.

command:

• Active backup save will block user operations
• The asynchronous mode of background backup bgsave does not block the persistence operation.

3-About persistent configuration

• save 900 1 Within 900 seconds, when the user performs an update operation, then it is persisted once
• save 300 10 Within 300 seconds, the user performed 10 update operations. Then it will persist once
• save 60 10000 Within 60 seconds, if the user performs 10,000 update operations, it will be persisted once
• save 1 1 Update and persist once within 1 second!! The performance is particularly low.
  

Find in the redis.conf configuration file

4-About persistent file name setting

默认的条件下,持久化文件名称 dump.rdb

5-File storage directory

./ 代表当前文件目录. 意义使用绝对路径的写法

---

Persistence Strategy in Redis-AOF

1-AOF features

• The AOF mode is closed by default. It needs to be turned on manually
• AOF mode records the user's operation process. Real-time persistence can be achieved. Ensure that data is not lost
• The persistent files maintained by the AOF mode take up a lot of space. So the persistence efficiency is not high. And the persistent files need to be maintained regularly
• Once the AOF mode is turned on, redis reads AOF files mainly in AOF mode

2-Turn on AOF mode

3-Persistence strategy

• always: user updates once, then persists once
• everysec: It is more efficient to persist once per second
• no: No active persistence. Operating system related. Almost not used

---

About Redis memory optimization strategy

Business scene

If you use redis frequently, keep saving data in it, and do not delete operations, the memory will inevitably overflow. Can the memory strategy be optimized. Can you automatically delete unused data and keep hot data in redis!!!

LRU algorithm

Introduction:

LRU is the abbreviation of Least Recently Used, that is, the least recently used. It is a commonly used page replacement algorithm. The page data that has not been used the most recently is selected to be eliminated. The algorithm gives each page an access field to record the time t elapsed since the last time a page was accessed. When a page needs to be eliminated, select the existing page with the largest t value, that is, the least recently used one The page is eliminated.

Calculated dimension: the elapsed time T since the last time.

Note: The LRU algorithm is the best algorithm used in memory optimization.

LFU algorithm

Introduction:

LFU (least frequently used (LFU) page-replacement algorithm). That is, the page replacement algorithm is the least frequently used, and the page with the smallest reference count is required to be replaced when the page is replaced, because the frequently used page should have a larger number of references. However, some pages are used a lot at the beginning, but will not be used in the future. Such pages will remain in the memory for a long time. Therefore, the reference count register can be shifted to the right by one bit at regular intervals to form an exponentially decayed average number of uses.

Dimension: Number of citations
Common sense: Move the computer to the left to expand the multiple, and the computer to the right to reduce the multiple

TTL algorithm

Description:

Sort the remaining survival time, and delete the data that will be deleted soon.

Redis default memory optimization strategy

Description 1:

The strategy adopted in Redis is regular deletion + lazy deletion strategy

Description 2:

• Periodic deletion: Redis checks whether there is an expired key every 100ms by default, and checks it in a random manner during the check. (Not all data is checked, because the efficiency is too low.)

• • Problem: Due to the large amount of data, it may not be selected when extracting. It may appear that the data has reached the timeout period, but redis did not delete the data immediately.

• Lazy strategy: When the user obtains the key, first check whether the data has passed the timeout period. If it has timed out, delete the data.

• • Problem: Due to the large amount of data, it is impossible for the user to get all the memory data once. The phenomenon that the data that needs to be deleted is inevitably kept in the memory will occupy memory resources

• The above-mentioned memory optimization methods can be used to actively delete.

Memory optimization algorithm description:

• Volatile-lru uses the LRU algorithm to optimize the data with the set timeout period
• allkeys-lru uses LRU algorithm to optimize all data
• Volatile-lfu uses the LFU algorithm to optimize the data with a set timeout period
• allkeys-lfu uses LFU algorithm for optimization in all data
• Volatile-random uses a random algorithm for data with a set timeout period
• allkeys-random All data uses random algorithm
• volatile-ttl TTl algorithm for setting timeout time
• noeviction does not actively delete data, and returns with an error if the memory overflows.
  

---

Consistent hash algorithm

Algorithm introduction

The consistent hash algorithm was proposed by the Massachusetts Institute of Technology in 1997. It is a special hash algorithm that aims to solve the problem of distributed caching. [1] When removing or adding a server, the mapping relationship between the existing service request and the processing request server can be changed as little as possible. Consistent hashing solves the problems of dynamic scaling of simple hash algorithms in the Distributed Hash Table (DHT) [2].

effect: 解决缓存数据,在哪存储的问题…

Algorithm description

Taste:

• If the data is the same, the hash result must be the same
• Common hash values ​​are composed of 8-digit hexadecimal numbers. How many possibilities are shared? 2^32
  

1-balance

平衡性是指hash的结果应该平均分配到各个节点，这样从算法上解决了负载均衡问题

Description:

Since the nodes are calculated through arithmetic calculations, the load may be unbalanced.

Data
balance is achieved through virtual nodes. Balance : 尽可能保证节点数据量均衡
virtual nodes:为数据量少的节点净夺数据

2-Monotonicity

单调性是指在新增或者删减节点时，不影响系统正常运行

Principle: If nodes are added/reduced, the original data should be as unchanged as possible

3-Dispersibility

分散性是指数据应该分散地存放在分布式集群中的各个节点(节点自己可以有备份)，不必每个节点都存储所有的数据

Distribute data storage, even if the server goes down in the future, the impact will only be part of it, not all.

Proverb: Don't put eggs in a basket.

---