Redis from entry to abandonment series (5) ZSet
The example in this article is based on: 5.0.4 ZSet is a relatively complex data structure in Redis. When the storage size is within 128 and the member length is less than 64, it is implemented as zipList, and if it exceeds, it is SkipList
Suddenly I found that the fifth article has not mentioned zipList yet. However, the previous chapters such as Hash and List all involve zipList, and I will write a separate zipList implementation later. Please look forward to [Redis from entry to Abandonment Series (Side Story) ZipList】
Closer to home, let's first take a look at how to use the ZSet type in redis
//将一个或多个元素及其分数加入到有序集合里面
ZADD key [NX|XX] [CH] [INCR] score member [score member ...]
Code example:
//添加元素
>zadd store 1000 xiaoming 2000 xiaoqiang 3000 xiaoyue
(integer) 3
//返回指定区间内的有序集合列表
> zrange store 0 -1 withscores
1) "xiaoming"
2) "1000"
3) "xiaoqiang"
4) "2000"
5) "xiaoyue"
6) "3000"
//返回有序集合的数量
>zcard store
(integer) 3
//查看处于1000到2000的存款的人数
>zcount store 1000 2000
(integer) 2
//查询处于1000到2000的存款的人群
> ZRANGEBYSCORE store 1000 2000
1) "xiaoming"
2) "xiaoqiang"
//根据member查看当前排名
>zrank store xiaoming
(integer) 0
So far, the usage of redis zset has come to an end.
Source code analysis
By convention, first come the data structure of zset
/* ZSETs use a specialized version of Skiplists */
typedef struct zskiplistNode {
sds ele;
double score;
struct zskiplistNode *backward;
struct zskiplistLevel {
struct zskiplistNode *forward;
unsigned long span;
} level[];
} zskiplistNode;
typedef struct zskiplist {
struct zskiplistNode *header, *tail;
unsigned long length;
int level;
} zskiplist;
typedef struct zset {
dict *dict;
zskiplist *zsl;
} zset;
The bottom layer of the ordered set encoded by SkipList is composed of a structure named zset, which has two data types, dict and zskiplist. Zskiplist saves all set elements according to the score from small to large, dict saves the mapping relationship between members and scores, and the two data structures share the memory of ele and score of the same elements. zskiplist is a doubly linked list, which is to facilitate the retrieval of elements in a range in reverse order. For the explanation of the jump list, please refer to the comic algorithm: What is a jump table?
How is redis implemented when we are using zadd key memberit? Let's take a look at the source code:
/* Insert a new node in the skiplist. Assumes the element does not already
* exist (up to the caller to enforce that). The skiplist takes ownership
* of the passed SDS string 'ele'. */
zskiplistNode *zslInsert(zskiplist *zsl, double score, sds ele) {
zskiplistNode *update[ZSKIPLIST_MAXLEVEL], *x;
unsigned int rank[ZSKIPLIST_MAXLEVEL];
int i, level;
serverAssert(!isnan(score));
x = zsl->header;
for (i = zsl->level-1; i >= 0; i--) {
/* store rank that is crossed to reach the insert position */
rank[i] = i == (zsl->level-1) ? 0 : rank[i+1];
while (x->level[i].forward &&
(x->level[i].forward->score < score ||
(x->level[i].forward->score == score &&
sdscmp(x->level[i].forward->ele,ele) < 0)))
{
rank[i] += x->level[i].span;
x = x->level[i].forward;
}
update[i] = x;
}
/* we assume the element is not already inside, since we allow duplicated
* scores, reinserting the same element should never happen since the
* caller of zslInsert() should test in the hash table if the element is
* already inside or not. */
level = zslRandomLevel();
if (level > zsl->level) {
for (i = zsl->level; i < level; i++) {
rank[i] = 0;
update[i] = zsl->header;
update[i]->level[i].span = zsl->length;
}
zsl->level = level;
}
x = zslCreateNode(level,score,ele);
for (i = 0; i < level; i++) {
x->level[i].forward = update[i]->level[i].forward;
update[i]->level[i].forward = x;
/* update span covered by update[i] as x is inserted here */
x->level[i].span = update[i]->level[i].span - (rank[0] - rank[i]);
update[i]->level[i].span = (rank[0] - rank[i]) + 1;
}
/* increment span for untouched levels */
for (i = level; i < zsl->level; i++) {
update[i]->level[i].span++;
}
x->backward = (update[0] == zsl->header) ? NULL : update[0];
if (x->level[0].forward)
x->level[0].forward->backward = x;
else
zsl->tail = x;
zsl->length++;
return x;
}
The above process is represented by a diagram, as shown below:
How is zset implemented when we are using zrank key memberit? Let's take a look at the source code
long zsetRank(robj *zobj, sds ele, int reverse) {
unsigned long llen;
unsigned long rank;
llen = zsetLength(zobj);
if (zobj->encoding == OBJ_ENCODING_ZIPLIST) {
//忽略掉 zipList查找过程
} else if (zobj->encoding == OBJ_ENCODING_SKIPLIST) {
zset *zs = zobj->ptr;
zskiplist *zsl = zs->zsl;
dictEntry *de;
double score;
de = dictFind(zs->dict,ele);
if (de != NULL) {
score = *(double*)dictGetVal(de);
rank = zslGetRank(zsl,score,ele);
/* Existing elements always have a rank. */
serverAssert(rank != 0);
if (reverse)
return llen-rank;
else
return rank-1;
} else {
return -1;
}
} else {
serverPanic("Unknown sorted set encoding");
}
}
/* Find the rank for an element by both score and key.
* Returns 0 when the element cannot be found, rank otherwise.
* Note that the rank is 1-based due to the span of zsl->header to the
* first element. */
unsigned long zslGetRank(zskiplist *zsl, double score, sds ele) {
zskiplistNode *x;
unsigned long rank = 0;
int i;
x = zsl->header;
for (i = zsl->level-1; i >= 0; i--) {
while (x->level[i].forward &&
(x->level[i].forward->score < score ||
(x->level[i].forward->score == score &&
sdscmp(x->level[i].forward->ele,ele) <= 0))) {
rank += x->level[i].span;
x = x->level[i].forward;
}
/* x might be equal to zsl->header, so test if obj is non-NULL */
if (x->ele && sdscmp(x->ele,ele) == 0) {
return rank;
}
}
return 0;
}
In fact, there are many places to enjoy the above insertion process when searching, and the ranking of users is obtained by accumulating spans.
Application scenarios
1. Leaderboard
2. Store social relationships
3. Sliding window application
