set
Redis Set is an unordered collection of type string.
The collection is realized through a hash table, so the complexity of adding, deleting, and searching are all O (1).
- New storage requirements: store large amounts of data and provide higher efficiency in query
- Required storage structure: able to save large amounts of data, efficient internal storage mechanism, easy to query
- Set type: It is exactly the same as the hash storage structure. It only stores the key and does not store the value (nil), and the value is not allowed to be repeated.
Basic commands
| command | Features |
|---|---|
| sadd key member1 [member2......] | adding data |
| smembers key | Get all data |
| srem key member1 [member2] | delete data |
| scard key | Get the aggregate data |
| sismember key member | Determine whether the collection contains the specified data |
> sadd students zhangsan lisi wangwu
(integer) 3
> smembers students
1) "zhangsan"
2) "wangwu"
3) "lisi"
> srem students zhangsan
(integer) 1
> scard students
(integer) 2
> sismember students zhangsan
(integer) 0
Extended Command 1
| command | Features |
|---|---|
| srandmember key [count] | Randomly get the specified amount of data in the collection |
| spop key [count] | Randomly get some data in the collection and move the data out of the collection |
srandmember students
"lisi"
> srandmember students 2
1) "wangwu"
2) "lisi"
> spop students
"wangwu"
Extended Command 2
| command | Features |
|---|---|
| sinter key1 [key2] | Find the intersection of two sets |
| sunion key1 [key2] | Find the union of two sets |
| sdiff key1 [key2] | Find the difference between two sets, front-back |
| sinterstore destination key1 [key2] | Find the intersection of two sets and store them in the specified set |
| sunionstore destination key1 [key2] | Find the union of two collections and store them in the specified collection |
| sdiffstore destination key1 [key2] | Find the difference between two sets and store it in the specified set |
| smove source destination member | Move the specified data from the original collection to the target collection |
> sunion students teacher
1) "zhangsan"
2) "lisi"
3) "yuwen"
4) "shuxue"
Application scenario
Business scenario 1
Each user will set up three hobbies when using Toutiao for the first time, but in order to increase the user's activity and interest points later, users must gradually become interested in other information categories and increase customer retention.
Business analysis
- The system analyzes the latest or hottest information items of each category and organizes them into set collections
- Randomly select some of the information
- Cooperate with the user's attention to the hot information in the information classification and organize into a full set of information displayed
solution
- Randomly get the specified amount of data in the collection
- Randomly get some data in the collection and move the data out of the collection
Business scenario 2
In order to promote communication between users and ensure the increase of business unit rate, each user needs to have a large number of friends. In fact, newcomers in the workplace do not
have more friends in the workplace. How to quickly accumulate more friends for users?
In order to increase user popularity and increase user retention, Sina Weibo needs Weibo users to pay attention to more people to obtain more information or hot
topics. How to increase the total number of users paying attention to others?
solution
- Find the intersection, union and difference of two sets
- Find the intersection, union and difference of two sets and store them in the specified set
- Move the specified data from the original collection to the target collection
Business scenario 3
The group company has a total of 12,000 employees, more than 700 roles in the internal OA system, more than 3000 business operations, more than 23,000 kinds of data, each
employee has one or more roles, how to quickly verify the authority of business operations
The company promotes its new website and counts the PV (visit), UV (independent visitor), and IP (independent IP) of the website.
solution
- Use the data de-duplication feature of set collection to record various access data
- Create string data and use incr to count daily visits (PV)
- Set model to record the number of different cookies (UV)
- Establish a set model to record the number of different IPs (IP)
Business scenario 4
Blacklists
Information information websites pursue high traffic, but because of the value of their information, they are often easily used by criminals. Through crawler technology, they can
quickly obtain information. Information from individual special industry websites can be converted into commercial secrets after being analyzed by crawlers For sale. For example, third-party train
tickets, airline tickets, hotel ticket purchasing software, e-commerce review comments, reviews.
At the same time, the fake traffic brought by the crawler will also give the operator the illusion and make the wrong decision, and effectively prevent the website from being crawled by the crawler repeatedly as
the basic problem that each website must consider. After distinguishing crawler users based on technical level, such users need to be effectively shielded. This is
the typical application of blacklist.
ps: It's not that crawlers must do destructive work. Some small websites need crawlers to bring some traffic to them.
Whitelist
For more secure application access, the blacklist alone cannot solve the security problem. At this time, it is necessary to set an accessible user group and
rely on the whitelist for more demanding access verification.
solution
- Set problem user discovery and identification rules based on business strategy
- Periodically update the user blacklist that meets the rules and add to the set collection
- After the user behavior information is reached, it is compared with the blacklist to confirm the behavior
- Blacklist filtering of IP addresses: information sources applied to open visitor access rights
- Blacklist filtering device information: information source used to restrict access to devices
- Blacklist filtering users: applied to information sources based on access rights
set application
-
Redis is used for random recommendation information retrieval, such as hot song list recommendation, hot news recommendation, hot travel route, application APP recommendation, big V recommendation, etc.
-
redis is used for related search of similar information, second-degree related search, and deep-related search.
Show common attention (once),
show common friends (once)
from user A, and get friend information list of friend B (once)
from user A , Get the shopping list list (second degree) of friend user B
from user A, get the game recharge list of friend user B (second degree) -
Redis applies to the fast deduplication of the same type of data
-
redis applies to service control based on blacklist and whitelist settings
Precautions
- The set type does not allow duplicate data. If the added data already exists in the set, only one copy will be kept
- set is the same as the storage structure of the hash, but the space for storing the value in the hash cannot be enabled
sorted_set
Redis zset, like set, is also a collection of string type elements, and does not allow duplicate members.
The difference is that each element is associated with a double type score. Redis uses the score to sort the members of the set from small to large. The members of zset are unique, but the score can be repeated.
- New storage requirements: data sorting is conducive to the effective display of data, and it is necessary to provide a way to sort according to its own characteristics
- Required storage structure: a new storage model that can save sortable data
- sorted_set type: add sortable fields based on the set storage structure
Basic commands
| command | Features |
|---|---|
| zadd key score1 member1 [score2 member2...] | adding data |
| zrange key start stop [WITHSCORES] | Sort in ascending order to get all the data in the specified ranking interval |
| zrevrange key start stop [WITHSCORES] | Sort in descending order to get all the data in the specified ranking interval |
| zrem key member [member ...] | Delete specified data |
| zrangebyscore key min max [WITHSCORES] [LIMIT] | Sort in ascending order to get data in the specified score interval |
| zrevrangebyscore key max min [WITHSCORES] | Sort in descending order ... |
| zremrangebyrank key start stop | Remove all data in the specified rank range from the ordered set |
| zremrangebyscore key min max | Remove all data in the specified score interval from the ordered set |
| zcard key | Get the total amount of collection data |
| zcount key min max | Get the total amount of data in the specified score interval of the collection |
| zinterstore destination numkeys key [key ...] | Merge the intersection of two ordered sets into a new ordered set |
| zunionstore destination numkeys key [key ...] | Merge the union of two ordered sets into a new ordered set |
> zadd zs 10 fir 8 sec 4 thi
(integer) 3
> zrange zs 0 2
1) "thi"
2) "sec"
3) "fir"
> zrevrange zs 0 -1 withscores
1) "fir"
2) "10"
3) "sec"
4) "8"
5) "thi"
6) "7"
//> zrem zs thi
//(integer) 1
> zrangebyscore zs 7 10 withscores
1) "sec"
2) "8"
3) "fir"
4) "10"
Extended commands
| command | Features |
|---|---|
| zrank key member | Get the index (rank) corresponding to the data, in ascending order |
| zrevrank key member | Get the index (rank) corresponding to the data, in descending order |
| zscore key member | Get the score of the specified data |
| zincrby key increment member | Increase for specified (can be negative, can be decimal) |
> zrank zs thi
(integer) 0
> zrevrank zs sec
(integer) 1
> zscore zs fir
"10"
> zincrby zs 5 fir
"15"
> zincrby zs -4.5 fir
"10.5"
Application scenario
Business scenario 1
Vote for the top ten outstanding young people in Guangdong, vote in various variety shows and sea elections.
Various resource websites TOP10 (movies, songs, documents, e-commerce, games, etc.)
chat room activity statistics
Game friend intimacy
Business analysis
- Establish a ranking basis for all resources participating in the ranking
solution
- Get the index (rank) corresponding to the data
- score value acquisition and modification
Business scenario 2
Basic service + value-added service websites will set up trial for members to let users fully experience the advantages of members. For example, watch movie trial VIP, game
VIP experience, cloud disk download experience VIP, data viewing experience VIP. When the VIP experience expires, if such information is effectively managed. Even for formal
VIP users, there are corresponding management methods.
The website will regularly open voting and discussion, and will be conducted within a limited time, and will be voided after the deadline. How to effectively manage such expired information.
solution
-
For the task processing based on the timeline limitation, record the processing time as the score value, and use the sorting function to distinguish the processing order
-
Record the next time to be processed, process the corresponding task when it is due, remove the record in redis, and record the next time to be processed
-
When a new task is added, determine and update the time of the next task to be processed
-
为提升sorted_set的性能，通常将任务根据特征存储成若干个sorted_set。例如1小时内， 1天内，周内，
月内，季内，年度等，操作时逐级提升，将即将操作的若干个任务纳入到1小时内处理的队列中 -
获取当前系统时间
time
业务场景3
任务/消息权重设定应用
当任务或者消息待处理，形成了任务队列或消息队列时，对于高优先级的任务要保障对其优先处理，如
何实现任务权重管理。
解决方案
- 对于带有权重的任务，优先处理权重高的任务，采用score记录权重即可
多条件任务权重设定
如果权重条件过多时，需要对排序score值进行处理，保障score值能够兼容2条件或者多条件，例如外贸
订单优先于国内订单，总裁订单优先于员工订单，经理订单优先于员工订单 - 因score长度受限，需要对数据进行截断处理，尤其是时间设置为小时或分钟级即可（折算后）
- 先设定订单类别，后设定订单发起角色类别，整体score长度必须是统一的，不足位补0。第一排序规则首
位不得是0 - 例如外贸101，国内102，经理004，员工008。
- 员工下的外贸单score值为101008（优先）
- 经理下的国内单score值为102004
zset的应用
- redis 应用于计数器组合排序功能对应的排名
- redis 应用于定时任务执行顺序管理或任务过期管理
- redis 应用于即时任务/消息队列执行管理
- redis 应用于限时按次结算的服务控制
注意事项
- score保存的数据存储空间是64位，如果是整数范围是-9007199254740992~9007199254740992
- score保存的数据也可以是一个双精度的double值，基于双精度浮点数的特征，可能会丢失精度，使用时
候要慎重 - The underlying storage of sorted_set is based on the set structure, so the data cannot be repeated. If the same data is added repeatedly, the score value will be
overwritten repeatedly, retaining the last modified result